Skip to main content

Full text of "A scale for the measurement of quality in English composition by young people"

See other formats


m 



W 






DC/J^k- 



LIBRARY 

Walter E. Femald 
State School 




Waverley, Massachusetts 

No . 304 ( 




>^\\^ 



9,^ 



•^^r- 




A SCALE FOR THE MEASUREMENT OF 

QUALITY IN ENGLISH COMPOSITION 

BY YOUNG PEOPLE 



By MILO B. HILLEGAS, PH.D. 

Assistant Professor of Elementary Education, Teachers College 
Columbia University 



Reprint of Teachers College Record, Vol. 13, No. 4, September, 1912 



THIRD IMPRESSION, FEBRUARY 1916 



PUBLISHED BY 

NEW YORK CITY 
1913 



Copyright, 1912, 1913, by Teachers College, Columbia University 



ACKNOWLEDGMENTS 

At the completion of the study here presented, I find myself 
deeply obligated to many individuals. The task of judging the 
sample compositions was not always an agreeable one, and the 
time necessary to do this work was no small matter with busy 
people; yet five hundred and fifteen individuals were willing to 
assist by grading one or the other of the sets of compositions, 
and thereby made possible the derivation of the scale. The 
cooperation of many individuals was due to the generous interest 
manifested by the editors of Science and The Journal of Educor- 
tional Psychology in bringing the study to the attention of their 
readers. 

I acknowledge my greatest obligation and deepest gratitude to 
Dr. E. L. Thomdike with whose advice and encouraging assist- 
ance this work was undertaken and conducted. 

M. B. H. 



CONTENTS 

Section Page 

1. Introduction. (A brief summary of the work that has 

been done on standards) ..... i 

2. The Scale. (The scale printed) .... 5 

3. What the Scale Measures 13 

4. The Location of the Zero Point for English Com- 

position 16 

5. Theory of the Method Employed . . . .18 

6. Sample Compositions from which the Scale was 

Derived . . . . . . . .26 

7. Results Obtained with the First Set of Samples 

AND THE Selection of the Second Set . - '2.7 

8. Results Obtained with the Second Set of Samples 32 

9. Sufficiency of Judges 56 

10. The Quality of Judges 57 

11. The Use of Mixed Types of Compositions . . 60 



SCALE FOR THE MEASUREMENT OF QUALITY 
IN ENGLISH COMPOSITION BY YOUNG PEOPLE 

Section i. Introduction 

Every attempt to measure the efficiency of instruction in a 
school system or to evaluate different methods of educational 
procedure serves to emphasize the importance of standards. 
Proper standards would make it possible to compare with cer- 
tainty the work done in one school or system of schools with 
that done elsewhere. They would make it more difficult for 
mere opinion to control so much of our school-room practice. 
After examining the arithmetical abilities of many pupils in dif- 
ferent representative school systems, Dr. C. W. Stone ('08) 
says : ^* Probably the truest single expression of the findings of 
this study is summed up in the word diversity. . . . Free- 
dom and initiative are here seen to have led educational prac- 
tice in widely varying paths. Certain paths are those of legiti- 
mate differentiation, but others are waste. . . . The great- 
est need shown by this research is standard of achievement. 
That the great variability herein shown would exist if school 
authorities possessed adequate means of measuring products is 
inconceivable, . . . ."^ 

Standards are greatly needed in order that we may define 
educational requirements. Results that are accepted in one col- 
lege or school as worthy of a mark of *' good," " seventy-five," 
or any other of the common conventional ratings, may or may 
not receive the same credit in another educational system. Such 
differences in the acceptance of results are seldom warranted 
by the variations in the statements of requirements issued by the 
institutions concerned. Indeed, as wide divergence in the ac- 
ceptance of results often exists between schools of the same 

* Arithmetical Abilities and Some Factors Determining Them, Colum- 
bia University Contributions to Education, Teachers College Series, No. 
19. 

I 



2 Measurement of Quality in English Composition 

system, and not infrequently it is found between different de- 
partments in the same institution. 

If there were standards or scales for the measurements of 
results in the various school subjects that would approximate 
the accuracy of the scales used in measuring extension, weight 
and time, educational administrators and investigators would be 
able to measure and express the efficiency of a school system in 
terms that would carry conviction. Such standards would also 
make it possible to define exactly the requirements of the col- 
lege, high school, and civil service: and by them school superin- 
tendents could define requirements for promotions from one 
grade to the next. 

The more or less satisfactory standards or scales that have 
been established in certain school subjects show that the task, 
while difficult, is by no means impossible. Dr. J. M. Rice 
C97)^ tested the spelling ability of some 3,000 public school 
pupils in 21 different school systems. His purpose was to com- 
pare the conditions existing in the various schools and to de- 
termine which influenced the spelling ability of the pupils. In 
order to do this he developed a series of tests which were given 
in each of the schools under as nearly the same conditions as 
possible. When the returns from these tests were scored, the 
tabulated results furnished a standard with which any teacher 
might compare the work of a class in spelling. 

Dr. O. P. Cornman ('02)^ made a more intensive study of the 
same character in several of the Philadelphia schools. He used 
some of the Rice tests and also developed others. The tests 
were uniformly scored and the results were tabulated in proper 
form for making comparisons between the various schools. 

Dr. Rice ('02)* also tested the arithmetical abilities of about 
6,000 school children in 17 schools in 7 cities. The methods 
employed in this research were similar to those employed in the 
spelling investigation. 



'The Futility of the Spelling Grind, The Forum, Vol. 23, pp. 163-172 
and 409-419. 

* Spelling in the Elementary School ; An Experimental and Statistical 
Investigation. 

* Educational Research : A Test in Arithmetic, The Forum, Vol. 34, 
pp. 281-297. Causes of Success and Failure in Arithmetic, The Forum, 
Vol. 34, pp. 437-452. 



Introduction 3 

Dr. Stone ('08)® tested the arithmetical abihties in funda- 
mentals and reasoning of some 6,000 school children in the 6A 
(high 6th) grade in 26 representative public school systems. 
Two of the problems on which this study has specific bearing 
are: "(i) What is the nature of the product of the first six 
years of arithmetic work? (2) What is the relation between 
distinctive procedures in arithmetic work and the resulting 
abilities?" A carefully differentiated test in the fundamentals 
was used and a separate test was given for reasoning abilities. 
As was done in all the other investigations, the results were 
uniformly scored, only in this case greater effort was made to 
evaluate the different problems. The results are to be found 
in tabular form. Dr. Stone says of his work, " It is believed 
that the present study will help to standardize the work in 
arithmetic in the first six grades. Anyone who wishes may 
know how his system or school compares with the representative 
systems of the country." The author gives explicit directions 
for the use of the tests, the method of scoring the results, and 
the use of the tables in comparing results. Subsequently these 
tests have been tried in a number of places, and the results 
have served to indicate the relative efficiency of the different 
schools. 

The whole process of developing such standards is well shown 
in the work which Mr. S. A. Courtis ('11)^ is conducting. He 
has developed an improved set of arithmetic tests by which it is 
claimed any grade in the school may be tested. These have been 
given to many thousands of children under uniform conditions. 
The results, when properly tabulated, will form a scale by which 
it will be possible to measure the achievement of any class or 
pupil. 

The feature of the studies which makes their consideration 
relevant to this study is not the conclusions reached but the 
fact that in each case tests were given to a sufficient number of 

" Arithmetical Abilities and Some Factors Determining Them, Columbia 
University Contributions to Education. Teachers College Series, No. 19. 

" Measurements of Growth and Efficiency in Arithmetic, Elementary 
School Teacher, Vol. 10, pp. 58-74 and 177-199. Elementary School 
Teacher, Vol. 12, pp. 127-137. Journal Educational Psychology, Vol. 2, 
p. 272. _ The Courtis Standard Tests in Arithmetic, with Manual of 
Instructions for giving and scoring. 



4 Measurement of Quality in English Composition 

pupils to enable the authors to establish a standard or scale with 
which they were able to measure the relative efficiency of the 
various systems. 

Dr. E. L. Thorndike ('lo)^ has published a scale for the 
measurement of quality in the handwriting of children and also 
one for the handwriting of women. The derivation of these 
scales involved a somewhat different procedure from that em- 
ployed for the scales in spelling and arithmetic. Many sam- 
ples both of children's handwriting and of adults' were secured 
and these were judged separately by many competent indi- 
viduals. From the scores thus obtained the relative values of 
the samples were determined and the scales formed. The 
scales in this case consist of a series of samples of hand- 
writing with which the samples that are to be measured are 
compared. In many particulars the methods employed in de- 
riving these scales in handwriting are the same as those used in 
the present study, and frequent reference will be made to Dr. 
Thorndike's study in the descriptions which follow. 

'Handwriting, Teachers College Record, Vol. ii, No. 2 [reprinted]. 



Section 2. The Scale 

This scale for the measurement of quaHty in English compo- 
sition by young people has been derived by methods that are 
described in the following pages. 

Sample 380. Value o. Artificial sample. 

Letter. 
Dear Sir: I write to say that it aint a square deal Schools 
is I say they is I went to a school, red and gree green and 
brown aint it hito bit I say he don't know his business not to- 
day nor yeaterday and you know it and I want Jennie to get 
me out. 

Sample 5P§. Value 183. Artificial sample. 

My Favorite Book, 
the book I refer to read is Ichabod Crane, it is an grate book 
and I like to rede it. Ichabod Crame was a man and a man 
wrote a book and it is called Ichabod Crane i like it because the 
man called it ichabod crane when I read it for it is such a great 
book. 

Sample 618. Value 260. Artificial sample. 

The Advantage of Tyranny. 
Advantage evils are things of tyranny and there are many ad- 
vantage evils. One thing it that when they opress the people 
they suffer awful I think it is a terrible thing when they say 
that you can be hanged down or trodden down without mercy 
and the tyranny does what they want there was tyrans in the 
revolutionary war and so they throwed off the yok. 

Sample Q4. Value 369. Written by a boy in the second year 

of the high school, aged 14 years. 

Sulla as a Tyrant. 

When Sulla came back from his conquest Marius had put 
himself consul so sulla with the army he had with him in his 
conquest siezed the government from Marius and put himself in 
consul and had a list of his enemys printy and the men whoes 
names were on this list we beheaded. 

5 



The Scale 7 

Sample 519. Value 474. Written by a girl in the third year 
of the high school, aged 17 years. 

De Quincy. 

First: De Quincys mother was a beautiful women and 
through her De Quincy inhereted much of his genius. 

His running away from school enfluenced him much as he 
roamed through the woods, valleys and his mind became very 
meditative. 

The greatest enfluence of De Quincy's life was the opium 
habit. If it was not for this habit it is doubtful whether we 
would now be reading his writings. 

His companions during his college course and even before 
that time were great enfluences. The surroundings of De 
Quincy were enfluences. Not only De Quincy's habit of opium 
but other habits which were peculiar to his life. 

His marriage to the woman which he did not especially care 
for. 

The many well educated and noteworthy friends of De 
Quincy. 



Sample 334. Value 585. Written by a boy in the fourth year 
of the high school, aged 16 years. 

Fluellen. 

The passages given show the following characteristic of 
Fluellen: his inclination to brag, his professed knowledge of 
History, his complaining character, his great patriotism, pride 
of his leader, admired honesty, revengeful, love of fun and 
punishment of those who deserve it. 



Sample 196. Value 675. Written by a girl in the first year 
of the high school, aged 18 years. 

Ichabod Crane. 

Ichabod Crane was a schoolmaster in a place called Sleepy 
Hollow. He was tall and slim with broad shoulders, long arms 
that dangled far below his coat sleeves. His feet looked as if 
they might easily have been used for shovels. His nose was 
long and his entire frame was most loosely hung to-gether. 



The Scale 9 

Sample 221. Value 772. Written by a boy in the third year of 
the high school, aged 16 years. 

Going Down with Victory. 

As we road down Lombard Street, we saw flags waving from 
nearly every window. I surely felt proud that day to be the 
driver of the gaily decorated coach. Again and again we were 
cheered as we drove slowly to the postmasters, to await the 
coming of his majestie's mail. There wasn't one of the gaily 
bedecked coaches that could have compared with ours, in my 
estimation. So with waving flags and fluttering hearts we 
waited for the coming of the mail and the expected tidings of 
victory. 

When at last it did arrive the postmaster began to quickly 
sort the bundles, we waited anxiously. Immediately upon re- 
ceiving our bundles, I lashed the horses and they responded with 
a jump. Out into the country we drove at reckless speed — 
everywhere spreading like wildfire the news, "Victory !" The ex- 
ileration that we all felt was shared with the horses. Up and down 
grade and over bridges, we drove at breakneck speed and 
spreading the news at every hamlet with that one cry *' Vic- 
tory ! " When at last we were back home again, it was with 
the hope that we should have another ride some day with " Vic- 
tory." 



Sample 571. Value 838. Written by a boy in the Freshman 

class in college. 

Venus of Melos. 

In looking at this statue we think, not of wisdom, or power, 
or force, but just of beauty. She stands resting the weight 
of her body on one foot, and advancing the other (left) with 
knee bent. The posture causes the figure to sway slightly to 
one side, describing a fine curved line. The lower limbs are 
draped but the upper part of the body is uncovered. (The un- 
fortunate loss of the statue's arms prevents a positive knowl- 
edge of its original attitude.) The eyes are partly closed, hav- 
ing something of a dreamy languor. The nose is perfectly cut, 
the mouth and chin are moulded in adorable curves. Yet to say 
that every feature is of faultless perfection is but cold praise. 
No analysis can convey the sense of her peerless beauty. 



The Scale 1 1 

Sample 177. Value 937. Written by a boy in the Freshman 

class in college. 

A Foreigner's Tribute to Joan of Arc. 

Joan of Arc, worn out by the suffering that was thrust upon 
her, nethertheless appeared with a brave mien before the Bishop 
of Beauvais. She knew, had always known that she must die 
when her mission was fulfilled and death held no terrors for 
her. To all the bishop's questions she answered firmly and with- 
out hesitation. The bishop failed to confuse her and at last 
condemned her to death for heresy, bidding her recant if she 
would live. She refused and was lead to prison, from there to 
death. 

While the flames were writhing around her she bade the old 
bishop who stood by her to move away or he would be injured. 
Her last thought was of others and De Quincy says, that recant 
was no more in her mind than on her lips. She died as she 
lived, with a prayer on her lips and listening to the voices that 
had whispered to her so often. 

The heroism of Joan of Arc was wonderful. We do not 
know what form her great patriotism took or how far it really 
led her. She spoke of hearing voices and of seeing visions. We 
only know that she resolved to save her country, knowing 
though she did so, it would cost her her life. Yet she never 
hesitated. She was uneducated save for the lessons taught her 
by nature. Yet she led armies and crowned the dauphin, king 
of France. She was only a girl, yet she could silence a great 
bishop by words that came from her heart and from her faith. 
She was only a woman, yet she could die as bravely as any 
martyr who had gone before. 



Section 3. What the Scale Measures 

The scale is composed of sample English compositions, the 
qualities of which have been determined by more than four 
hundred competent judges. The values which are assigned to 
the various samples express their quality in the same sense, 
though not as accurately, as millimeters express the lengths of 
Hues. Just as 183 mm. may be expressed by 18.3 cm. or 1.83 
dm., so the 183 employed in this scale may be considered as 
183 small units of quality, or as 18.3 units ten times as large, 
or as 1.83 units one hundred times as large. The values here 
employed should not be confused with the customary per cents 
that are used in rating English compositions. A difference of 
two hundred in this scale is equal to twice a difference of one 
hundred taken in any part of the scale. Thus sample 94 with 
a value of 369 is a little more than twice as good an English 
composition as sample 595 with a value of 183. Sample 519 
with a value of 474 is about one-half as good as sample 177 with 
a value of 937. 

Merit in English writing is complex. Judges are influenced 
both by form and by content. Such factors of form as spell- 
ing, punctuation, capitalization, and the like are subject to defi- 
nite rules. Form is, therefore, more easily measured than con- 
tent. When an individual is in doubt concerning the relative 
merits of two English compositions, the tendency is to fix upon 
one or more of the obvious form elements and for the time 
being to give them undue importance in fixing the relation of 
the samples. 

No attempt has been made in this study to define merit. 
The term as here used means just that quality which competent 
persons commonly consider as merit, and the scale measures 
just this quality. The accompanying graphic representation of 
the location of the various samples composing the scale will 
serve to make plain the meaning of the scale. 

The value of any English composition may be obtained 
by placing it beside the samples constituting the scale and 
determining to which it most nearly corresponds. The 
use of the scale will be clearly understood, if the reader 

13 



14 Measurement of Quality in English Composition 

will compare sample 376, given below, with the scale. 
There will be no hesitation in pronouncing sample 376 
better than the lower end of the scale until quality 585 
is reached. The majority of persons will consider sample 376 , 



9 uo zoo ioo 100 m eoo 709 soo 900 /poo 
S80 S9S- m n s/9 ^'«?f m zv ^7/ nj 

Fig. I. Graphic representation of the relative values of the samples com- 
posing the scale. The numbers above the line represent the various 
unit points in the scale. The numbers below the line represent sample 
compositions and show their relative positions on the scale. Thus, 
sample 580 is at the zero point, sample 595 is between the values 
of 100 and 200, etc. 



worth about yy2 or the equal of sample 221. In an actual test 
with seventy-three individuals, seventeen considered sample 376 
inferior to sample 534, sixteen thought it inferior to sample 196, 
and twenty-one thought sample 376 was better than sample 571 ; 
but there was almost perfect agreement that sample 376 was 
nearly if not quite equal to sample 221. If the reader thinks 
sample 2t7^ is better than sample 221 but not as good as sample 
571, he may place the value between yy2 and 838. By this 
method the value of any sample may be expressed as accurately 
as the individual cares to make it. *' The sample to be measured 
should, for convenience, be examined with the entire scale in 
view. If the scale's samples are arranged in order on a table or 
against a wall, the examined sample is easily compared with them. 
The measurer then decides what quality of the scale the sample 
possesses and records the measure. . . . The measure may 
be made more and more accurate by having other judges also 
measure, each always in ignorance of the ratings given by the 
others. In default of other judges, the measure may be made 
more accurate by rating the sample two or more times, each 
time in ignorance of the ratings previously given. An individual 
may be measured more accurately by using several samples of 
his compositions, each being rated in ignorance of the ratings 
given to the other samples."® 



Handwriting, p. 8. 



What the Scale Measures 15 

Sample j/d. The Military Career of Alexander the Great. 

Alexander the Great one of the greatest generals of the 
world began his military career at the age of twenty. It was 
then at his father's death that he crushed all rebellion both in 
Macedonia and Greece. Then he started on his famous march. 
Crossing the Hellespont he defeated the Persians at the river 
Granicus from thence he followed the coast of Asia Minor. 
However he turned aside from his course and going up into 
Phrygium he cut the Gordian knot. Next he defeated a large 
Persian army at Issus, captured Tyre after a long siege and 
much labor. He now went into Egypt, founded Alexandria, 
and returned to Persia. At Arbela the Persian army was an- 
nihilated and Darius fled. Alexander next conquered Babylon 
and Persepolis and then having wandered thru eastern Persia 
he crossed the Hindu Hush Mountains and entered India. Al- 
though victorious Alexander's army was weary of such hard 
travelling and refused to go farther. So the great general was 
forced to lead them back to Babylon and his military career was 
finished. 



Section 4. The Location of the Zero Point in English 

Composition^ 

The location of the zero point for merit in English composi- 
tion by young people is of importance in order to allow the 
" times as much merit " judgment. Measures of fatigue, prac- 
tice, or change of any sort will be greatly facilitated if we can 
so arrange the scale that 8, 10, and 12 on it mean twice as far 
from just not any of the thing in question as 4, 5, and 6; 
that 240 means twice as much of the thing as 120, three times 
as much as 80, five times as much as 48. Hitherto no one could 
say with any assurance whether sample 571 was two, ten or a 
hundred times as " good " as 618. 

It has been located approximately as sample 580 on the basis 
of the judgments of (i) nine men of special literary ability, five 
of them professors of English, of whom four are also authors 
of standard text-books on English composition, and four 
men of marked general intellect and literary ability, (2) 
eleven gifted teachers familiar with secondary education, and 
(3) eight psychologists familiar with the significance of scales 
and zero points in the case of intellectual abilities and products. 

Although no one person of these had any deliberate criterion 
for the point where positive merit just begins, and although 
one's first reaction to the request to locate such a point is to 
regard it as arbitrary, there is much agreement among indi- 
viduals and almost perfect agreement in the case of the averages 
of the three groups. 

These individuals located the zero point in connection with 
the set of 27 samples^^ either by placing it below the worst of 
these or by stating which of these they considered to possess ab- 
solutely no merit as specimens of English composition by young 
people. 

Of the twenty-eight judges, two regard zero merit as some- 
thing below the least meritorious specimens of this list, and 
five put it higher than sample 618, but three-fourths of the 

" For the experimentation and calculation in connection with the loca- 
tion of the zero point. Professor E. L. Thorndike is responsible. 
"For the samples used in this work see Section 8. 

16 



Location of the Zero Point 



17 



judges locate it as lower than sample 607 or higher than sample 
595. The central tendency of the twenty-eight puts it at or just 
barely below specimen 580. 

Taking the sample regarded as just barely of some merit and 
that regarded as the next below it in the case of each judge, we 
have the following results: 



Below 607 


4 times 


" 607 


9 " 


" 670 


2 " 


58 


3 " 


" 501 


6 " 


" 491 


3 " 



Below 580 


5 times 


39 


3 « 


79 


6 « 


" 595 


2 " 


" 603 


2 " 


" 618 


2 « 


Above 618 


9 « 



The median opinion of these 28 judges thus places the begin- 
ning of merit in English writing by young people at sample 580. 



Section 5. Theory of the Method Employed in Deriving 

THE Scale 

Nearly five hundred individuals judged one or the other of 
the sets which contained the samples used in the scale. Each 
person was asked to arrange the samples in order of merit. 
This amounted to a request that each sample should be com- 
pared with every other sample in the set. In scoring the results, 
the poorest sample was numbered one, the next poorest two, and 
so on; thus in a set of eighty-three samples the best was num- 
bered eighty-three. The scale was derived from these scores by 
the method of right and wrong cases. ^^ The theory of this 
method as applied to this study may be stated as follows : Dif- 
ferences that are equally often noticed are equal, unless the 
differences are either always or never noticed.^^ 

Merit in English writing is the resultant of a large number of 
independent factors. Among the individuals who judged the 
samples used in this study, several greatly undervalued certain 
samples because they thought that the language used was too 
mature for high school pupils. On the other hand, several of 
the artificial samples were greatly overvalued because the judges 
thought that they detected childish modes of expression in 
these. When a large number of individuals are required to 
judge the quality of any sample English composition, many will 
be nearly correct, some will be in error because of overvaluing it, 
and an equal number will undervalue it. Small errors of judg- 
ment will be much more common than large ones. The theory of 
probability enables us to construct a diagram that will represent 
what would result if a large number of persons were to judge 
an English composition, or what would result if one person 
were to judge independently a large number of times. Figure 

11 shows the probable grouping of judgments, when a large 
number of individuals are requested to determine the value of 
a composition. The line XO represents diagrammatically the 
value of an English composition. The bell-shaped surface, 
ABC, shows the distribution of judgments. The shape of the 

" Fullerton and Cattel, On the Perception of Small Differences, pp. 

12 ff. 

" Handwriting, p. 5. 

18 



Theory of the Method Employed 



19 



curve will depend upon the accuracy of the judges. The better 
the judges the higher that part of the curve above O and the 
shorter the base AC. The surface ABC represents diagram- 
matically a normal surface of distribution. The line BO divides 
the surface into two equal parts. PQ and MN are equally 
distant from BO, and within the area PMNQ, fifty per cent 
of the judgments are found.^^ The line OP or its equal OM 
represents an error of such size that half of the errors are larger 
and half are smaller. It is the median error or as it is called 
the median deviation (M. D.)^* 




Fig. II. A normal surface of frequency on which is shown the distri- 
bution of errors in judgment when a large number of individuals are 
required to determine the value of a sample English composition 
whose value by consensus of opinion is represented by the line XO. 



In Fig. Ill, the values of two samples are represented, one 
by the line XO and the other by the line X'O'. Such a difference 
in value is taken that the line O'P', which represents this dif- 
ference, is equal to the median deviation. The area P'C'Q' 
contains seventy-five per cent of the judgments of the sample 
represented by X'O'. Seventy-five per cent, therefore, of the 
judgments are correct and twenty-five per cent (the area 
A'P'Q') are wrong, because they make the sample represented 
by XO better than the sample represented by X'O'. When 
seventy-five per cent of the judges notice the difference in 
quality between two samples, the difference is equal to the 
median deviation, and the ratio between the difference and the 
median deviation in this case is i/i or i. Throughout this 
study this ratio is taken as the unit of value. The unit may be 



"For a complete statement of the relations that exist between the 
various parts of a normal surface of distribution, see Thorndike, Mental 
and Social Measurements, p. 59. 

" The name * probable error ' is applied to this value. 



20 Measurement of Quality in English Composition 

defined as that difference in quality which exactly seventy-five 
per cent of the judges observe. 

In Figs. IV and V the arrangement of the curves is such as 
to show the relation that exists between the difference and the 
median deviation when various percentages of the judges notice 
the difference. Curves No. i and No. 2 show this relation when 




Fig. III. Two surfaces of frequencies so placed as to show the difference 
in value of two samples of English compositions when seventy-five 
per cent of the judges consider one better than the other. 



fifty per cent of the judges believe that sample represented by 
curve No. 2 is better than sample represented by curve No. i. 
In this case BO extended passes through point O' and there 
is no difference in the quality of these samples. Curve No. 3 
is so placed that fifty-five per cent of the judges consider sample 
represented by curve No. 3 better than sample represented by 
curve No. i. The difference is represented by the line Y'O''. 
By actual measurement Y'O'' is tVo of P"0''. Curve No. 4 
shows sixty per cent " better " judgments. The difference 
Y'^'O'" is T^ of the median deviation (P"'O"0- In curve No. 
5 the percentage of " better " judgments is sixty-five and the 
difference is irV of the median deviation. The following table 
shows the curve compared with curve No. i, the percentage of 
better judgments represented by the curve, and the ratio between 
the difference in value and the median deviation (^^^^). Median 
deviation is defined on page 19. 



Theory of the Method Employed 



21 





TABLE I 




The Ratios Between the Differences in Value and the Median Devia- 


tion Corresponding to Certain Percentages of "Better" 


Judgments as Shown by the Curves in Figs. Ill, IV and V 






Percentage 


Difference divided by 




Curve compared 


of "better" 


the median deviation 




with curve No. 1 


judgments 


(D/M. D.) 


Fig. IV 


No. 2 


50 





« 


No. 3 


55 


.19 


<( 


No. 4 


60 


.38 


u 


No. 5 


65 


.57 


u 


No. 6 


70 


.78 


Fig. Ill 


No. 2 


75 


1.00 


Fig. V 


No. 7 


80 


1.25 


(i 


No. 8 


85 


1.53 


(( 


No. 9 


90 


1.89 


u 


No. 10 


95 


2.43 


u 


No. 11 


100 


? 



A table giving the differences for all percentages from fifty 
to ninety-nine is found on page i6 of " On the Perception of 
Small Differences." This table is also found on page 164 of 
" Mental and Social Measurements." In selecting the samples 
that should be used in the scale, it was more convenient to begin 
with the poorer samples, so the table was transmuted to give 
the differences for percentages from i to 50, and the differences 
for tenths of a per cent were interpolated. The results are 
found in Table II. 

Any standard or scale should be based on an unit such that 
equal units may be derived independently of the scale. The 
unit in this scale has been defined as that difference which 
seventy-five per cent of the judges are able to distinguish. All 
that is required to derive this unit is a set of samples that vary 
from each other by small degrees in quality. When two sam- 
ples are found such that seventy-five per cent of the judges 
agree in calling one better than the other, the difference is just 
the difference used as the unit in this scale. 



22 Measurement of Quality in English Composition 

B 




No. 1. 



No. 2. 



No. 3. 



No. 4. 



No. 5. 



No. 6. 



0*^ M" 

Fig. IV. Various frequency curves so placed in relation to curve number 
one as to show the difference in value between two samples when 
the percentage of "better" judgments is either 50, 55, 60, 65, or 70 
respectively. 



Theory of the Method Employed 



23 



No. 1. 



No. 7. 



No. 8. 



No. 9. 



No. 10. 



No.ll. 




Fig. V. Various frequency curves so placed in relation to curve number 
one as to show the difference in value between two samples when 
the percentage of "better" judgment is either 80, 85, 90, 95, or 100 
respectively. 



24 Measurement of Quality in English Composition 



TABLE II 

Table for Determining the Median Deviation from the Percentage 
OF Wrong Cases and the Amount of Difference 

The table reads as follows: When i per cent of the judges regard 
sample A worse than sample B, the difference, A-B, is 3.45 of the median 
deviation. When i.i per cent of the judges regard sample A worse 
than sample B, the difference, A-B, is 3.41 of the median deviation. When 
1.2 per cent of the judges regard sample A worse than sample B, the 
difference, A-B, is 3.37 of the median deviation, etc. 



icntage 
worse ' ' 
laments 


D 


Percentage 

of " worse " 

judgments 


D 


Percentage 

of "worse" 

judgments 


D 


Percentage 

of "worse" 

judgments 


D 


Percentage 

of "worse" 

judgments 


D 


^ ^ 


M. D. 


M. D. 


M. D. 


M. D. 


M. D. 


1.0 


3.45 


6.0 


2.31 


11.0 


1.82 


16.0 


1.47 


21.0 


1.20 


.1 


3.41 


.1 


2.30 


.1 




81 


.1 


1.46 


.1 


1.19 


.2 


3.37 


.2 


2.29 


.2 




80 


.2 


1.46 


.2 


1.19 


.3 


3.33 


.3 


2.27 


.3 




80 


.3 


1.45 


.3 


1.18 


.4 


3.29 


.4 


2.26 


.4 




79 


.4 


1.45 


.4 


1.18 


.5 


3.25 


.5 


2.25 


.5 




78 


.5 


1.44 


.5 


1.17 


.6 


3.21 


.6 


2.24 


.6 




77 


.6 


1.43 


.6 


1.16 


.7 


3.17 


.7 


2.23 


.7 




76 


.7 


1.43 


.7 


1.16 


.8 


3.13 


.8 


2.21 


.8 




76 


.8 


1.42 


.8 


1.15 


.9 


3.09 


.9 


2.20 


.9 




75 


.9 


1.42 


.9 


1.15 


2.0 


3.05 


7.0 


2.19 


12.0 




74 


17.0 


1.41 


22.0 


1.14 


.1 


3.02 


.1 


2.18 


.1 




73 


.1 


1.41 


.1 


1.14 


.2 


3.00 


.2 


2.17 


.2 




73 


.2 


1.40 


.2 


1.13 


.3 


2.97 


.3 


2.16 


3 




72 


.3 


1.40 


.3 


1.13 


.4 


2.95 


.4 


2.15 


.4 




71 


.4 


1.39 


.4 


1.12 


.5 


2.92 


.5 


2.14 


.5 




71 


.5 


1.39 


.5 


1.12 


.6 


2.89 


.6 


2.12 


.6 




70 


.6 


1.38 


.6 


1.12 


.7 


2.87 


.7 


2.11 


.7 




69 


.7 


1.38 


.7 


1.11 


.8 


2.84 


.8 


2.10 


.8 




68 


.8 


1.37 


.8 


1.11 


.9 


2.82 


.9 


2.09 


.9 




68 


.9 


1.37 


.9 


1.10 


3.0 


2.79 


8.0 


2.08 


13.0 




67 


18.0 


1.36 


23.0 


1.10 


.1 


2.77 


.1 


2.07 


.1 




66 


.1 


1.35 


.1 


1.10 


.2 


2.75 


.2 


2.06 


.2 




66 


.2 


1.35 


.2 


1.09 


.3 


2.73 


.3 


2.05 


.3 




65 


.3 


1.34 


.3 


1.09 


.4 


2.71 


.4 


2.04 


.4 


1 

-*• 


64 


.4 


1.34 


.4 


1.08 


.5 


2.70 


.5 


2.03 


.5 




64 


.5 


1.33 


5 


1.08 


.6 


2.68 


.6 


2.03 


.6 




63 


.6 


1.32 


.6 


1.07 


.7 


2.66 


.7 


2.02 


.7 




62 


.7 


1.32 


.7 


1.07 


.8 


2.64 


.8 


2.01 


.8 




61 


.8 


1.31 


.8 


1.06 


.9 


2.62 


.9 


2.00 


.9 




61 


.9 


1.31 


.9 


1.06 


4.0 


2.60 


9.0 


1.99 


14.0 




60 


19.0 


1.30 


24.0 


1.05 


.1 


2.58 


.1 


1.98 


.1 




59 


.1 


1.30 


.1 


1.05 


.2 


2.57 


.2 


1.97 


.2 




59 


2 


1.29 


.2 


1.04 


.3 


2.55 


.3 


1.96 


.3 




58 


.3 


1.29 


.3 


1.04 


.4 


2.54 


.4 


1.95 


.4 




58 


.4 


1.28 


.4 


1.03 


.5 


2.52 


.5 


1.95 


.5 




57 


.5 


1.28 


.5 


1.03 


.6 


2.50 


.6 


1.94 


.6 




56 


.6 


1.27 


.6 


1.02 


.7 


2.49 


.7 


1.93 


.7 




56 


.7 


1.27 


.7 


1.02 


.8 


2.47 


.8 


1.92 


.8 




55 


.8 


1.26 


.8 


1.01 


.9 


2.46 


.9 


1.91 


.9 




55 


.9 


1.26 


.9 


1.01 


5.0 


2.44 


10.0 


1.90 


15.0 




54 


20.0 


1.25 


25.0 


1.00 


.1 


2.43 


.1 


1.88 


.1 




53 


.1 


1.25 


.1 


1.00 


.2 


2.41 


.2 


1.88 


.2 




53 


.2 


1.24 


.2 


.99 


.3 


2.40 


.3 


1.87 


.3 




52 


.3 


1.24 


.3 


.99 


4 


2.39 


.4 


1.86 


.4 




51 


.4 


1.23 


.4 


.98 


.5 


2.38 


.5 


1.85 


.5 




51 


.5 


1.23 


.5 


.98 


.6 


2.36 


.6 


1.85 


.6 




50 


.6 


1.22 


.6 


.97 


7 


2.35 


.7 


1.84 


.7 




49 


.7 


1.22 


.7 


.97 


.8 


2.34 


.8 


1.83 


.8 




48 


.8 


1.21 


.8 


.96 


.9 


2.32 


.9 


1.83 


.9 




.48 


.9 


1.21 


.9 


.96 



Theory of the Method Employed 



25 



ientage 
worse " 
pnents 


D 


1 
Percentage 
of " worse " 
judgments 


D 


Percentage 

of "worse" 

judgments 


D 


Percentage 

of "worse" 

judgments 


D 


Percentage 

of " worse " 

judgments 


D 


Q_j *^^ .*«. 


M. D. 


M. D. 


M. D. 


M. D. 


M. D. 


26.0 


.95 


31.0 


.74 


36.0 


.53 


41.0 


.34 


46.0 


.15 


.1 


.95 


.1 


.74 


.1 


.53 


.1 


.34 


.1 


.15 


.2 


.94 


.2 


.73 


.2 


.52 


.2 


.33 


.2 


.14 


3 


.94 


.3 


.73 


.3 


.52 


.3 


.33 


.3 


.14 


.4 


.93 


.4 


.72 


.4 


.51 


.4 


.32 


.4 


.13 


.5 


.93 


.5 


.72 


.5 


.51 


.5 


.32 


.5 


.13 


.6 


.93 


.6 


.71 


.6 


.51 


.6 


.32 


.6 


.13 


.7 


.92 


.7 


.71 


.7 


.50 


.7 


.31 


.7 


.12 


.8 


.92 


.8 


.70 


.8 


.50 


.8 


.31 


.8 


.12 


.9 


.91 


.9 


.70 


.9 


.49 


.9 


.30 


.9 


.11 


27.0 


.91 


32.0 


.69 


37.0 


.49 


42.0 


.30 


27.0 


.11 


.1 


.91 


.1 


.69 


.1 


.49 


.1 


.30 


.1 


.11 


.2 


.90 


.2 


.68 


.2 


.48 


.2 


.29 


.2 


.10 


.3 


.90 


.3 


.68 


.3 


.48 


.3 


.29 


.3 


.10 


.4 


.89 


.4 


.67 


.4 


.47 


.4 


.28 


.4 


.09 


.5 


.89 


.5 


.67 


.5 


.47 


.5 


.28 


.5 


.09 


.6 


.88 


.6 


.67 


.6 


.47 


.6 


.28 


.6 


.09 


.7 


.88 


.7 


.66 


.7 


.46 


.7 


.27 


.7 


.08 


.8 


.87 


.8 


.66 


.8 


.46 


.8 


.27 


.8 


.08 


.9 


.87 


.9 


.65 


.9 


.45 


.9 


.26 


.9 


.07 


28.0 


.86 


33.0 


.65 


38.0 


.45 


43.0 


.26 


48.0 


.07 


.1 


.86 


.1 


.65 


.1 


.45 


.1 


.26 


.1 


.07 


.2 


.85 


.2 


.64 


.2 


.44 


.2 


.25 


.2 


.06 


.3 


.85 


.3 


.64 


.3 


.44 


.3 


.25 


.3 


.06 


.4 


.84 


.4 


.63 


.4 


.43 


.4 


.24 


.4 


.06 


.5 


.84 


.5 


.63 


.5 


.43 


.5 


.24 


.5 


.06 


.6 


.84 


.6 


.63 


.6 


.43 


.6 


.24 


.6 


.05 


.7 


.83 


.7 


.62 


.7 


.42 


.7 


.23 


.7 


.05 


.8 


.83 


.8 


.62 


.8 


.42 


.8 


.23 


.8 


.05 


.9 


.82 


.9 


.61 


.9 


.41 


.9 


.22 


.9 


.04 


29.0 


.82 


34.0 


.61 


39.0 


.41 


44.0 


.22 


49.0 


.04 


.1 


.82 


.1 


.61 


.1 


.41 


.1 


.22 


.1 


.04 


.2 


.81 


.2 


.60 


.2 


.40 


.2 


.21 


.2 


.03 


.3 


.81 


.3 


.60 


.3 


.40 


.3 


.21 


.3 


.03 


.4 


.80 


.4 


.59 


.4 


.40 


.4 


.21 


.4 


.02 


.5 


.80 


.5 


.59 


.5 


.40 


.5 


.21 


.5 


.02 


.6 


.80 


.6 


.59 


.6 


.39 


.6 


.20 


.6 


.02 


.7 


.79 


.7 


.58 


.7 


.39 


.7 


.20 


.7 


.01 


.8 


.79 


.8 


.58 


.8 


.39 


.8 


.20 


.8 


.01 


.9 


.78 


.9 


.57 


.9 


.38 


.9 


.19 


.9 


.00 


30.0 


.78 


35.0 


.57 


40.0 


.38 


45.0 


.19 


50.0 


.00 


.1 


.78 


.1 


.57 


.1 


.38 


.1 


.19 






.2 


.77 


.2 


.56 


.2 


.37 


.2 


.18 






.3 


.77 


.3 


.56 


.3 


.37 


.3 


.18 






.4 


.76 


.4 


.bb 


.4 


.36 


.4 


.17 






.5 


.76 


.5 


.55 


.5 


.36 


.5 


.17 






.6 


.76 


.6 


.55 


.6 


.36 


.6 


.17 






.7 


.75 


.7 


.54 


.7 


.35 


.7 


.16 






.8 


.75 


.8 


.54 


.8 


.35 


.8 


.16 






.9 


.74 


.9 


.53 


.9 


.34 


.9 


.15 







Section 6. The Sample Compositions from Which the 

Scale was Derived 

The first step in the derivation of the scale was the collec- 
tion of about seven thousand English compositions by young 
people. These were obtained from various sources and repre- 
sent a definite attempt to obtain particularly the very poorest 
and the best work that is done in the schools. After collect- 
ing these samples the author and one other graded them roughly 
into ten classes. From these classes seventy-five samples were 
selected. In order that the samples at the extremes of the scale 
might be measured, it was necessary to supply some artificial 
samples. The poorest of these were conscious attempts by adults 
to write very poor English. The best samples were obtained from 
youthful writings of such literary geniuses as Jane Austen and 
the Brontes. The works of some college Freshmen were also 
used. As finally constituted, the set consisted of eighty-three 
samples which varied from the poorest to the best by small 
degrees of quality. 

Nearly all of the samples were in the handwriting of the 
authors, some of which was very good and some very poor. 
In order that the character of the handwriting might not in- 
fluence the judges, all the samples were typewritten and mimeo- 
graphed. Great care was exercised in reproducing all of the 
mistakes that appeared in the originals. 

It would have been desirable to have included more than the 
eighty-three samples, but the persons whose services were par- 
ticularly desired were busy, and since the judging of a set of 
even eighty-three required from three to six hours of close 
application, it was not possible to enlarge the set. 

Separate sets of these samples were given to about one 
hundred individuals with the following printed request : " Please 
arrange these in order of merit as specimens of English com- 
position by young people. If there are two or more of them 
which seem to you to be absolutely equal in merit, give them 
the same number as a score. Score your results on the accom- 
panying sheet calling the worst specimen i, the next worst 2, 
the next worst 3, and so on.'^ 

26 



Section 7. The Results Obtained with the First Set 
OF Samples and the Selection of a Second Set 

More than one hundred individuals worked with the set of 
eighty- three samples. Some, however, did not understand the di- 
rections and some made such serious mistakes in recording their 
results that it was possible to use only seventy-three of the 
records. These records were copied on a large sheet for con- 
venience in handling, and the distribution of the positions as- 
signed to each sample was determined. The next step was to 
compare the position assigned each sample with that assigned 
each other sample in order to determine just how many of 
the judges had considered any given sample better than any 
other given sample. These " better " judgments were arranged 
in tabular form such as is shown in Table IV on page 30. 

There were great variations in the positions assigned to the 
samples. It was not possible to establish the position of any 
sample with reasonable accuracy until many more judgments 
could be secured. The judgments of the seventy-three indi- 
viduals indicated the samples that were probably nearly equal 
in merit, and in this way made it possible to select from the 
eighty-three samples a much smaller set which would still con- 
tain all the important steps in quality from the poorest to the 
best. The samples that were included in the new set are re- 
produced in Section 8, and reference to Table V, which shows 
the percentage of " better " judgments for the samples selected 
from the set of eighty-three, will help in understanding the 
method that was employed in selecting this set. 

All but three of the judges had agreed that sample 607 was 
the poorest. This sample was included, as well as the next 
samples in merit, 580 and 595. The remaining eighteen sam- 
ples were selected by taking the sample that about seventy-five 
per cent of the judges had agreed was better than the last 
sample selected. The differences in merit between samples 607 
and 580, and between samples 580 and 595 were large. New 
artificial samples were included with these three samples and 
these were judged by a number of individuals in order to de- 
termine several samples that would range in merit between the 

27 



28 Measurement of Quality in English Composition 

two pairs. In this way five were selected, making in all a set 
of twenty-seven samples. Later samples 519 and 520 were 
added. 

Since only twenty-three of the eighty-three samples in the 
first set are used in any succeeding set, only such data as bear 
upon these twenty-three samples are here given. Table III 
shows the distribution of positions assigned to these samples 
by seventy-four individuals. This table is followed by Table IV 
which gives the number of " better " judgments for this set, 
and by Table V which shows the percentage of " better '* judg- 
ments for each pair of samples. Only the significant data are 
given in Tables IV and V. 



Results Obtained from First Set of Samples 



29 



^ £ 
"^ o 

Oj o 

M ^ 

5a 

t>Co3 

«^ 

O S 
Oj G 

2° 

O) ^ 
^ OJ 

rt © 
§^ 
^^ 

0) GO 

» ^ CD 

o o o 

JO ^ fe"^ 

o -jf a 
^ ^^ 
•I ^ 

-M •• Oh 
^ OQ J3 

<U^ ^ 

^!=^— I 

^^ « 
M ?g «5 

O '=^'73 
8 ^, fl 

"^ 28 

OQ CD 



2 a 



Samples 


I-H 






(N<M (N 


to to to CO 


to CO CO 

I-H tH I-H 


00 
GO 




CO 


iH 1-H CO CO 


»H CO to CO Ci 


rH I-H 
rH 1-H OQ 


lit) 
CO 
10 






(N to 


1> CO (N 00 00 


CO CO 

r— ( I-H rH 


CO 






iH <N rH CO CO 


Tf (N CO CJi Oi 


1HCOO 
rH iH rH 







T— ( 1—1 


(M CO CO CO CO 


t>. t^ iH 00 rH 
rH rH 


TtHtO (N 


I— 1 
10 






rH iH (N >0 to 


00 00 to CO 
rH T-H 


to CO (N 







rH (N 


to TjH to 
tH 


coio -* ^ t>. 

rH 


to Tj* to 





CO 




(N 


rJ<CO TjH 05 CO 


rHOCOCO Tj^ 
tH tH 


Tti(NCO 


I-H 




1-H Ttl 


to (M Tt< !>. 

rH 


t^ "* TjH 
iH rH 


COCO 


CO 
CO 




1-H (N 


-^ I> Tf (N (M 

iH rH 


tOO5(N00(N 


CO tH (N 


CO 

Oi 

T-H 




T-H rH 1— 1 to C^ 


T-t rH iH 


tO(N to <N 


r-^ 1—^ 







CO COO 
rH 


05 rH to CO CO 


(N>0 (NrH tH 




CO 




CO CO OiO 

I-H 


I>t^00C0CO 

1—i 


CO »H CO CO 


r-^ 


CO 
10 




rH to TJ^^^CO 
rH 


»H to rH CO iH 
y—i rH 


to I-H iH rH tH 


rH 


05 
1— 1 


(M 


»0 CO COOO 

iH r-H rH 


00 Ttl "^ (N iH 




iH 


CO 


I-H CD 


CO CO 00O5 -^ 
i—tr-irH 


CO rH 








(N t-i(£) 


iH t> tH "^ 
iH iH (M rH 


(N 








rH COOO <-l 
1—1 


OI>l> -^ tH 

(N rH 


<N 






CO 



CO 


1-1 t>. 10 CO 

T-H CO 


"^ to CO iH rH 








00 
1—1 
CO 


I-H I— 1 t^ IC Ci 


to tH (N 

T-H 








10 

10 


rH to (N 10 


1H 









00 

10 


CO 


rH 









CO 


(M T-H rH 


saopi 


SOJ 


I-H <N CO "^10 


C0b-00 050 
rH 


rH (M CO Tt< to 
iH tH rH tH tH 


CO 1^00 050 

tH rH T-H tH C<1 


»H (N CO 
(N (N (N 



03 

CD 

ua 
c 

03 
V 

T3 

CO 

T3 

-3 
a 



30 Measurement of Quality in English Composition 



an 

< 
m 

w 

H 
I 

H 
o 






O) 

o 

00 






Pi 
W 

I 

M 
O 



^ O 

w o 
c 

Q 
P 



(4 
H 
H 

? 
O 
» 

IS 
P 

» 



t3 c3 
P « 

o c 

CI "^ 

l§ 

•• p 

^-^ 

O O) 



^ 1^ 

§^ 

rP 73 

CO c[j 

-I 

0) o 

3 



c3 



o 

a, 

a 

o 
o 



a, 



03 






00 
00 



CO 



CO 



O 






o 



o 

o 
CO 






CO 
CO 



CO 
C55 



O 
O 



CO 



CO 



Oi 
I— I 



(M 

CO 






o 



CO 

o 

CO 



00 
1—1 
CO 






o 

00 
U5 



73 

a <=^ 

o 



(N T-H CO (N b- Tt< O C^ »d^ COrJ< 
1-1 ^gs^,_i^c<l (NCO 



CO Tt^rfoOt^CO Oi CSi O 00 05 •-( 

I— It— I 1— It— ICq*— tC<l CO 



(MCOOOI>T-H lO (M (N (M CO 
T-H —I (M (M (N CO 



rt< T}4 Tt< (M id rHTj<(M«0 

T-l .— I rH (M C^ C^ 



I— I 00 T-l t>. -^ i-i Tt< (M b- 00 
i-H 1-1 (N (M (N CO CO 



CO COCOt-t^TtH 1-1 (N 

rH C^ (NCO 



1-iCO (M(MCOO000 (N 
T-i 1— I 1-1 (N iM (M 



rH 00 T-l 00 05 (NCO 

T— I 1—1 CO CO 



Tti (N O t^ 05 Tti 

1—1 T— I rH I— I CO 



»— I i—i CO lO ^ O CO 

T-l I— I T— I (N 



CO lO (N tH CO !>• 
1-1 (N (M (N 



<N lO -^ CO »0 
rH CO (N 



T-l T-l Ttl lO CO -^ 

(NCO 



Tt< 00 05 r-l 
(N 



T-l CO l> CO t- 



05 O 05 Tt< 
T— I T— I r^ 



CO 



(N 



1-1 CO CO 00(35 



O CO i-l O 
rH CO 



COfOi 



1— I (05 



l>OiO00C0 0'*I^05'^ (NOcOCOtH OO'-lO'^ »ooo 
00005T-IO 1^ 05 (N rH CO (N005(N(N O (N t^ (N CO «0 00 
CO lO lO CO CO »0 CO lO "5 CO C<l rH CO (N CO »0 ^ CI rt" ^O tJ< 



Results Obtained from First Set of Samples 



31 



o 

Pi 

a 

PQ 

ti 
o 



S 
H 

O 

tn 
H 

;?; 
o 

W 
Ph 

O 
H 



PQ Q 



Mi 
H 



H 

Q 

•-9 



M 
H 
H 



o 

pq 
P 





a 

a 
.2 

a 


M 

'a 

a 

02 


1> 

r-t 






t^T^f-KNlO 


050 CMCO to 


i-HOS 


CM rH rt< CO Ci 


00 t^ CO OCO 
rH CM rH CM CO 


to to 

CO rf 


00 

00 




1—1 


"^ Tt^ 000 CO 


l^t^ COCM 


01 

f-H 


10 to COftT^ 
1— i(M 1-t 


to to l^ Tt 05 
CM CM CM CM CO 


10 

CO 
10 






t^ rH OOlO Oi 


CO 00 00 00 t^ 




<N -^ 05 "^ 
rH t— i 


05 CTJ CDOO 
CM CM CM CM ''i^ 


CO 






TjH rti '^ (N CO 

to to to CO 
1-1 (N 


05 Ttt 00 rH 

rt< CM d to 
1-1 CO CM CO 









00 

d 
I— 1 


Oi to 05 TJH Tfl 

■>* oi 06 06 CM 
1—1 1— 1 CM CO 


000 '^ 

05 rH 

CM to to 




1-H 
10 




r— 1 


rH 1— 1 10 05 Ttl 

06 ^ C5 CM CM 
CM CO 


Tt< CM 

00 CO 

CM TJH 






10 




06 


CM CM CO 00 00 

CO coi> ^-^- 
1— * 1— 1 1— 1 CO CO 


00 

d 

CM 






CO 




00 

CD 
1—1 


C55O0 1>.CM CO 

Ttl to CO Tfi 
1-1 rH CM ■<* TtH 






1— 1 




TtH -^ Tt^ (M 

1— 1 1— < 1— 1 CO 
1—1 


COOI> 05 

d CO to to 

CM CM CM TtH 






CO 
CO 




Tt^ 00(MCO 
r-l CO rt^ d 


CO to i-H 

CO ^-i 

CM 1-1 CO 






CO 

T— I 




1-1 OOC^ TtH 

Tt CO CO 06 

1-1 <M 


1-1 to 

1-i d 

CO CO 









C^ 


I— ( 


!>. GO 05 CO 

C^ CO06 TfH 


00 

CO 

CO 






CO 


I— ( 


Tfl Ttl 00 i-H 05 

.-H to CO 1— 1 10 

CO TtH 








CO 

10 


1—1 


Tin 00 (N TtH 

»0 <N 00 
1-H 1— 1 C^J 








05 

T— I 




10 CO 1—1 

05 -— 1 CO 
(N CM 








CO 


(M to 
(M CO 

1— 1 T— 1 


t^ 1-1 

iOOO 










1.4 
14.9 
17.6 


00 









10 


r^ 1-1 T-t CO »>• 
rH T^l -tl Tf 10 


CO 



CO 


T-H 05 CO 
-^ TJH 

1-t Ttl 


00 

1— 1 
CO 


Ttt T-t l>. 
I-I TJH 10 


10 

10 


r-i (N 

1-H 



00 




0) 


3 


i> 0^ ooco 
CO 0: .-1 
CO ^ to CO CO 


"^ t^ 05 TtH 
1> 05 (M 1-1 CO 

iO CO to 10 


CM OCO COrH 

CM 05 CM CM 
CO C<J rH CO CM 


00 rH Tt^ 

OCMt^CMCO 
CO to to CM Tj< 


to 00 

000 

to -^ 



Section 8. Results Obtained with the Second Set of 

Samples 

The method employed in selecting the samples that were 
placed in the second set was given in the preceding section. 
The set is reproduced below except that samples 519 and 520 
were not included in all of the sets judged. 

Sample 60J, Artificial. 

Sketch. 

I words four and two came go billa guni sing hay cows and 
horses he done it good he died it goon I want yes sir yes sir 
oxes and sheeps he come yes sir camed and goes billum gumun 
oomunn goodum. 

Sample 4pi. Artificial. 

I say never mind. I say he knows all right all right dony 
puki I say it aint. I got one I got to noo and loo, lov and sov. 
Dont you care all right all right I say 

Sample 58. Artificial. 

i from thre ours up in room on books and books care for chil- 
der tore a page and rite on them 

Sample 501. Artificial. 

dere techer: 

I like schol not like schol. that man other place like make 
work tools. Some day you say I rede 

Sample 670. Artificial. 

I want to say it aint no youse they aint got no right and they 
aint got no man ban tan pan pan san san sem sen sun sun tun 
tun 

Sample 580. Artificial. 

Letter. 

Dear Sir: I write to say that it aint a square deal Schools 
is I say they is I went to a school, red and gree green and 

32 



Results Obtained from Second Set of Samples 33 

brown aint it hito bit I say he don't know his business not to- 
day nor yeaterday and you know it and I want Jennie to get 
me out. 



Sample p/. Artificial. 

I write to tell I like the job to work what mr Lambert said 
as he giv me a dolar harf a day to do things round his plase. 
I not aifrade to work so I say plees giv me the job and I can 
look after the cow all right or anything no trouble long time I 
work for mr. panter he knows 

Sample jp. Artificial. 

The man take little girls in big to the schul when it rains, the 
wagon with thing to keep warm and light up the rode all the 
way to Mis Gow. 

Sample 5^5. Artificial. 

My Favorite Book. 

the book I refer to read is Ichabod Crane, it is an grate book 
and I like to rede it. Ichabod Crame was a man and a man 
wrote a book and it is called Ichabod Crane i like it because the 
man called it ichabod crane when I read it for it is such a great 
book. 



Sample 618. Artificial. 

The Advantage of Tyranny. 

Advantage evils are things of tyranny and there are many 
advantage evils. One thing is that when they opress the people 
they sufifer awful I think it is a terrible thing when they say 
that you can be hanged down or trodden down without mercy 
and the tyranny does what they want there was tyrans in the 
revolutionary war and so they throwed oflf the yok. 

Sample 60^. Artificial 

A Character Sketch. 

The man I am describing is a white man and he has nice 
hair and wears a hat, and his horse is black, I like this man and 
he has two eyes and his nose is red. 



34 Measurement of Quality in English Composition 

Sample ^jo. Artificial. 

Description of School Room. 

Our school room is on the side of the school house and it 
is a awfully nice room and I like it because it is so nice and 
all the boys like it, and there is a good many pictures on the 
wall and there is a clock on the wall. We like this school room 
and come to school most all the time. 



Sample 62^. Artificial. 

A Scene. 

I think the sunlight is very beautiful on the water, and when 
it shines on the water it is very beautiful, and I love to watch 
it when it is so beautiful. The colors are so pretty and the noise 
of the water with the sunshine are so attractive in the sunshine 
I wonder do other people love to watch the water like I do. 
I dont know as there is anything as lovely as the water waves in 
the sunlight of the glorious orb. 



Sample P4. 

Sulla as a Tyrant. 

When Sulla came back from his conquest Marius had put 
himself consul so sulla with the army he had with him in his 
conquest siezed the government from Marius and put himself 
in consul and had a list of his enemys printy and the men whoes 
names were on this list we beheaded. 



Sample 200. 

My dear Fred, — 

I will tell you of my journey to Delphi Falls, N. Y. There 
is nice scenery along this route. The prettiest scene is in the 
glulf which is quite narrow, a small creek flows down it and 
the road follows along near its banks. 

There are woods on either side, these trees look very pretty 
when they are white with snow. 

In summer it is always shady and cool in them and the small 
fish may be seen darting back and forth in the water. 

I hope I will have the pleasure of taking you over the route 
some time. Yours sincerely, 



Results Obtained from Second Set of Samples 35 

Sample s^2. 

A Picture. 

I should like to see a picture, illustrating a part of L'allegro. 
Where the godesses of Mirth and Liberty trip along hand in 
hand. Two beautiful girls dressed in flowing garments, danc- 
ing along a flower-strewn path, through a pretty garden. Their 
hair flowing down in long curls. Their contenances showing 
their perfect freedom and happiness. Their arms extended 
gracefully smelling some sweet flower. In my mind this would 
make a beautiful picture. 

Sample 534, 

Fluellen. 

The passages given show the following characteristic of Flu- 
ellen: his inclination to brag, his professed knowledge of His- 
tory, his complaining character, his great patriotism, pride 
of his leader, admired honesty, revengeful, love of fun and 
punishment of those who deserve it. 



Sample igd. 

Ichabod Crane. 

Ichabod Crane was a schoolmaster in a place called Sleepy 
Hollow. He was tall and slim wath broad shoulders, long arms 
that dangled far below his coat sleeves. His feet looked as 
if they might easily have been used for shovels. His nose was 
long and his entire frame was most loosely hung to-gether. 

Sample 221. 

Going Down with Victory. 

As we road down Lombard Street, we saw flags waving from 
nearly every window. I surely felt proud that day to be the 
driver of the gaily decorated coach. Again and again we were 
cheered as we drove slowly to the postmasters, to await the 
coming of his majestie's mail. There wasn't one of the gaily 
bedecked coaches that could have compared with ours, in my 
estimation. So with waving flags and fluttering hearts we 
waited for the coming of the mail and the expected tidings of 
victory. 

When at last it did arrive the postmaster began to quickly 
sort the bundles, we waited anxiously. Immediately upon re- 



36 Measurement of Quality in English Composition 

ceiving our bundles, I lashed the horses and they responded 
with a jump. Out into the country we drove at reckless speed 
— everywhere spreading like wildfire the news, " Victory !" The 
exileration that we all felt was shared with the horses. Up 
and down grade and over bridges, we drove at breakneck speed 
and spreading the news at every hamlet with that one cry " Vic- 
tory ! " When at last we were back home again, it was with 
the hope that we should have another ride some day with ** Vic- 
tory." 



Sample ^00. 

The Preacher of Auburn. 

The most popular man of Auburn was the preacher. Al- 
though he had a very small salary he was contented. The 
preacher was kind to everybody. Little children loved him. Old 
soldiers liked to sit by his fireside and tell stories of the battles, 
which they had fought in. The beggars who came to his door, 
although chided for leading such an existence, were always 
clothed and feed. 

The preacher was always willing to go to the homes where 
there was sickness or death. Here he helped in all things that 
he could. 

In the church he preached with unaflFected grace, and all who 
came to scoff at him remained to worship. 

The minister was a contented, simple and kind man, whom 
the people loved. 

Sample j^j. Artificial. 

Essay on Burns. 

As far as I can learn from the Essay on Burns, Mr. Carlyle 
considers that good poetry must contain the sincerity of the 
poet. The poem must show the author's good choice of sub- 
ject and his clearness of sight. In order to have good poetry 
the poet must be familiar with his subject and his poem will 
show it. 

The characteristics of a great poet, in Mr. Carlyle's opinion 
were sincerity and choice of subject. A poet must be appre- 
ciative of nature and have a responding heart. Carlyle says a 
true poet does no have to write on subjects which are far away 
and probably come from the clouds. A truly great poet makes 
the most of subjects which are familiar to him and close to 
earth, as Burns did in his poems to the Field Mouse and The 
Daisey. 



Results Obtained from Second Set of Samples 37 

Sample ^65. Artificial. 

October. 

I can say without blushing that I think I was born in the best 
month of the year — October. There is something in a fair 
October day that makes me feel healthier and happier than any 
other time of the year. I think there are good reasons for this 
feeling. 

At what time of the year do we have such crisp mornings, 
such glorious noons and such soft cool twilights? All the 
sweetness of summer seems to be merging into the chill of 
winter. There is contentment in the atmosphere. The farmer 
is rejoicing over his crops ; the hunter is seeking satisfaction and 
pleasure in the forest; the business man is looking forward to 
the holiday trade; the college man, fresh from his vacation, is 
at his best ; and Nature herself is aflame with color. The whole 
world seems to be at peace with itself. It is the vacation time 
for Mother Earth, a lull between the torrid heat of summer and 
the cold blasts of winter. 

Sample 4^4, Artificial. 

A Diary. 

I had an early run in the woods before the dew was off the 
grass. The moss was like velvet and as I ran under the arches 
of yellow and red leaves I sang for joy, my heart was so bright 
and the world was so beautiful. I stopped at the end of the 
walk and saw the sunshine out over the wide " Virginia 
meadows." 

It seemed like going through a dark life or grave into heaven 
beyond. A very strange and solemn feeling came over me as 
I stood there, with no sound but the rustle of the pines, no one 
near me, and the sun so glorious, as for me alone. It seemed 
as if I had felt God as I never did before, and I prayed in my 
heart that I might keep that happy sense of nearness all my 
life. 

Sample ij^. Artificial. 

A Foreigner's Tribute to Joan of Arc. 

Joan of Arc, worn out by the suffering that was thrust upon 
her, nevertheless appeared with a brave mien before the Bishop 
of Beauvais. She knew, had always known that she must die 
when her mission was fulfilled and death held no terrors for 
her. To all the bishop's questions she answered firmly and 
without hesitation. The bishop failed to confuse her and at 



38 Measurement of Quality in English Composition 

last condemned her to death for heresy, bidding her recant if 
she would live. She refused and was lead to prison, from there 
to death. 

While the flames were writhing around her she bade the old 
bishop who stood by her to move away or he would be injured. 
Her last thought was of others and De Quincy says, that re- 
cant was no more in her mind than on her lips. She died as 
she lived, with a prayer on her lips and listening to the voices 
that had whispered to her so often. 

The heroism of Joan of Arc was wonderful. We do not 
know what form her great patriotism took or how far it really 
led her. She spoke of hearing voices and of seeing visions. We 
only know that she resolved to save her country, knowing 
though she did so, it would cost her her life. Yet she never 
hesitated. She was uneducated save for the lessons taught her 
by nature. Yet she led armies and crowned the dauphin, king 
of France. She was only a girl, yet she could silence a great 
bishop by words that came from her heart and from her faith. 
She was only a woman, yet she could die as bravely as any 
martyr who had gone before. 

Sample 220. 

Going Down with Victory. 

I sat on the top of a mail-coach in Lombard street impatiently 
atwaiting the start. 'Twas the night of the victory and we 
would help spread the news over England. 

Up jumps the coachman followed by the guard, an instant's 
preparation, a touch of the lash and we are off! We are soon 
past the limits of the city out in open country, galloping, tear- 
ing along, a clear road ahead of us for the English Mail stops 
for nothing. 

We dash in at villages, stopping but a moment with the mail, 
shouting the news of the victory and we are off again. Proud 
were we and had we not a right to be? The first to carry the 
great news through the land ! 

The memory of that ride is ever fresh in my mind and I will 
ever remember those hours as the most glorious in all my life. 

Sample 488. Artificial. 

Letter. 

Your letter gave me real and heartfelt pleasure, mingled with 
no small share of astonishment. Mary had previously in- 
formed me of your departure for London, and I had not ven- 
tured to calculate on any communication from you while sur- 
rounded by the splendours and novelties of that great city, 



Results Obtained from Second Set of Samples 39 

which has been called the mercantile metropolis of Europe. 
Judging from human nature, I thought that a little country girl, 
for the first time in a situation so well calculated to excite 
curiosity and to distract attention, would lose all remembrance 
for a time at least, of distant and familiar objects, and give her- 
self up entirely to the facination of those scenes which were 
then presented to her view. Your kind, interesting and most 
welcome epistle showed me, however, that I had been both 
mistaken and uncharitable in these suppositions. I was greatly 
amused at the tone of nonchalance which you assumed while 
treating of London and its wonders. Did you not feel awed while 
gazing at St. Paul's and Westminister Abbey ? Had you no feel- 
ings of intense and ardent interest when in St. James you saw 
the palace where so many of England's kings have held their 
courts, and beheld the representations of their persons on the 
walls? You should not be too much afraid of appearing coun- 
try-bred ; the magnificance of London has drawn exclamations 
of astonishment from travelled men, inexperienced in the world, 
its wonders and beauties. Have you yet seen anything of the 
great personages whom the sitting of Parliment now detains in 
London — the Duke of Wellington, Sir Robert Peel, Earl Grey, 
Mr. Stanley, Mr. O'Connell? If I were you, I would not be too 
anxious to spend my time in reading whilst in town. Make use 
of your own eyes for the purpose of observation now, and, for 
a time at least, lay aside the spectacles with which authors 
would furnish us. 

Sample 577. 

Venus of Melos. 
In looking at this statue we think, not of wisdom, or power, 
or force, but just beauty. She stands resting the weight of 
her body on one foot, and advancing the other (left) with knee 
bent. The posture causes the figure to sway slightly to one side, 
describing a fine curved line. The lower limbs are draped but 
the upper part of the body is uncovered. (The unfortunate 
loss of the statute's arms prevents a positive knowledge of its 
original attitude.) The eyes are partly closed, having some- 
thing of a dreamy langour. The nose is perfectly cut, the 
mouth and chin are moulded in adorable curves. Yet to say 
that every feature is of faultless perfection is but cold praise. 
No analysis can convey the sense of her peerless beauty. 

More than one hundred of the sets consisting of twenty-seven 
samples were mailed to individuals whose positions as teachers, 
authors, and literary workers implied that they were competent 



40 Measurement of Quality in English Composition 

judges of English writing. The following directions accom- 
panied each set : " Please arrange these samples in order of 
merit as specimens of English composition by young people. 
After determining your arrangement, pile all the specimens so 
that the best is on the bottom and the poorest on the top, and 
securely fasten them. Do not consult with anyone." 

When seventy-five replies had been received the results were 
tabulated in the same manner as in the case of the first set. 
Tables VI and VII give the same data for this set as are given 
in Tables III and IV for the first set of judges. Table VIII 
gives the per cent of " better " judgments and Table IX gives 
the differences in quality between the samples. 

Meanwhile Dr. E. L. Thorndike secured the judgments of 
forty-one individuals who were especially competent to judge 
merit in English writing. In order that these results might be 
used as a check on the others, they are tabulated separately in 
Tables X to XIV inclusive. 



Results Obtained from Second Set of Samples 41 





T— 1 








COrH 


toco CO rH 
rH 


OCO 






00 
00 








1— 1 C<l rH 


10 1^ to to "^ 

r-i 


OtO 

(N i-H 






CO 

10 






1— 1 


rH (N CO -"^ TjH 


CO to Tt< Oi 

rH 


to Tti 

rH r-^ 






CO 






1—1 


rH (M CO CO to 


TJH 05 i-H 

1—1 


05 <N 

rH 













(M 


(N CO(N 

rH 


00 CO CO CO 00 

1—1 rH 


to CO 












r-t 1—1 


CO to »H to 00 


t^ Tt^ (M 05 Tf 
rH 1—1 


COC^ 






CO 








(N(N too to 

1—1 


CO rH torxN 

1—1 r-i 


t>. rH 






1—1 






(N 


CO ^^ CO 00 CO 


OS 00 00 to iH 


(M CO 






CO 
CO 






I-H (M 


CO to (M 00O> 


rH COCO (N t^ 

i-H 








CO 

r— 1 






rH Tt^ t^ 


CO(N OiOtO 
1— 1 rH 1—1 rH 


rH (M rH 















(N rH 000 

I— ( 


t^ rH 00t> CO 


(N to -^ rj< 


CO 






CO 




(M 


T-H CO Tfl 
rH 


CO "^ t>. to "^ 

1—1 rH 


to rH COCO 






i 


CO 
10 






(N r-cooo 
1—1 


(M 00 to toco 

rH 


(M COrH 








(M 

CO 




rH (M 1-1 


to 00 I-H lO 

rH (N rH 


IXM (M rH 










05 




(N rHl> 


05 OCO(M to 

(N rH rH 


i—i 1—1 


1—1 













(MOO 00 


CO (M r^ t^ Ttl 

rH rH rH 


(M i-H 1-1 










CO 



CO 


?— 1 


1-1 CO CO 10 i-H 

1— ( r-t 


rH «3 GO (M 
rH rH 


1—1 










00 

1— ( 
CO 


(N 


rH r-H t*. 00 05 

1—1 


CO 00 Tt^ 
rH 1—1 


rH 










to 

Ci 

10 


1— 1 1— 1 


rt< 10 CO 00 CO 

tH i-H 1— i 


(N '^ I-H 
1—t 












CO 


1-H T-((N 


10 CO »o 1—1 CO 

fH (N rH 


-* (N (N 1-1 


rH T-i 










^ 


CO Tt< t>. 


00 UO 05 10 rH 
1-1 (M 


(N rH 














00 

10 


(N TJH TfH 
T— 1 


05I>010 CO 
iH rH rH 


rH 














CO 


i-H>l>. ICCO 


to -^ 1—1 1—1 

rH 


tH 












1—1 


10 


■«*< ^ Oi (N T-H 


CO rH 


rH 












00 

10 


CO (M 00 1-1 00 

rH (N 1— 1 1— 1 


i-H <N 














1—1 


00 Tt^ CO -^ CO 

1— ( rH 1— 1 I— 1 


CO CO tH 














1> 



CO 


00 00 (N (N CO 

CO rH 1—1 


rH 1—1 












suopi 


80J 


rH (M CO-^lCl 


COI> 00 Oi 

1— ( 


rH (N CO ''^ to 
rH r-i rH tH rH 


CO i^ 00 05 

1— ) fH rH rH C<1 


rH (MCO Tt< to 

(N<N <N(N(N 


CO h- 

(NC^ 





42 Measurement of Quality in English Composition 





05 

1—1 




i-H 




05 




rt^ 




« 


03 


ft 

s 


Ph 


03 


S 


CO 


< 


ti 


yj 


e3 




j3 


'A 


-M 


m 


t-i 


> 


<u 


QQ 




(H 


-Q 


g 


Sf^ 


w 


CO q; 


r i 


J^oo' 




p,io 


fLi 


Si? 


o 


M 


1^. § 


7) 


P m 






-< 




W 


'to r^ 


H 


G ^ 


»H 


O p, 


is 


O 0) 




-t-s 




OQ -M 


UQ 




h) 
-< 


§i 


ID 


'O T-H 


> 

Q 


•S-E 


I— ( 


p s 




> 03 


M 


y3 CO 


b 


>>'S 


t^ 


>3? 




5?; 


(D CO 


H 


CO fl 


> 
H 


_gS 


JJ 


-M l> 


O 

03 
H 


O ^ 


55 


a> 


U 




IS 
O 
Q 




1— V 


^ 5 




« c3 


^ 
p^ 

ti 




H 


'O fc. 


B 


oj (D 


U 




XI 


a>x5 




^ 


■• 


T— 1 


Pc< 


'^ a> 


o 


3^ 


ti 

H 


CO &, 


U4 


-^ 02 


!?: 


c3_^ 


^ 


-(J '73 


u 




w 


'^^ 


H 


h:s 




GQ 




a 




o 




u 



Samples with which comparison is made 


t— 1 






1-H 


Tf (N 00 1>CO 


CO CO to .-H 00 1-H 

rH (M i-H CO CO CO 


00 
00 






CO 


CO CO (N to 00 

rH 1-H tH 


rH O CO CO l> 

(M C^ (M CO -"ti 










rH CO 


to rH CO C^ -^ 
1-H 


O0I> 00 TtH 

(M rH (M 




CO 






I-H i-H to 


O Tft O 00 rH 
rH 1-H tH CM 


CO CO CO 
(M CO (N 




o 






1-H tH O 

rH 


Tf l> Oi CO o 

i-H I-H (M CO 


O rti 

CO ^ 




1— 1 






iH Tj^ 


O Tt^ (M O "^ 

T-^ rH (M (M 


CO 




O 

o 

CO 






tH rH rH iH 
i-H 


(M t^ (M Tti Tt^ 

T-< rH Ol CO 






1— 1 
CO 






I-H rH (M CO 

I-H 


CO Tti 05 CO 
I-H I-H I-H CO 






CO 

CO 




rH 


I-H 1-H CO 
r-i 


rH (M Ttl 
(M rH C^ 






CO 

Oi 
1— 1 




<M(N 


1-H CO rH (N fH 
rH (M 


CO 05 

CO CO 






o 
o 






(N CO to t^ 


CO 






CO 




--KN 


(M Tlt^ to rH rH 
rH CO 








CO 




CO 


CO -^ o to 

1-H 








CO 




<-( CO CO 05 
rH 


O rH C5 
C\J rH 












0(N O rf^ 

rH (M 










O 


I-H 


rH O (N lO t^ 


I-H 








CO 
o 

CO 


(M CO 


CO ^ t^ OI> 

rH Ol CO 










00 
I— 1 
CO 


1— 1 I— 1 CO rH 


lO CO CO ^ 
rH (M 












r-l rH (N lO CO 


00 00 (M 










CO 


CO t>- r-H O t^ 
I-H 


I-H (M 

CO CO 












rH OQ CO CO CO 


05 
rH 










o 

00 

to 


T-H CO CO CO CO 
rH 


o 

CO 


O 00 CO Tt* 

CQ Tti CO u:) 


t— ( 

o 

lO 


o ooo 

rH I— ( I— 1 


00 
lO 


rH TtH 


I—t 


CO 

rH 


pajBdi 
aidra 


JIOD 


t>. 1-1 00 1-H o 

005 lOO t^ 
CO""^ iOCO 


O Cni 0»0 00 
00 b-CO 05 I-H 
lO to CO 


CO Ort^ I>Tt< 
O t^ 05 (M CO 
CO to CO to 


(M OCOCO rH 

(N O C7i (M (N 
CO (N rH CO (M 


O rH O-^iO GO 
Ot^ (M CO CO 00 
CO to (N Tt* to ^ 



Results Obtained from Second Set of Samples 



43 



c 



Tj^ 



O ft 

n M 

S a 

wo 
(U CO 

©a 

^ (B 

«3 
f^ § 

o " 

Q '^ 



OQtJ 



Eh Q OJ 
W C JH 

:: ^^ 



5s —H C3 
O -3 .. 



--3 V 
tf S ►^ 



n 



CO cu 



cd 
a> 





a 
.2 

C 

CO 

'C 
c3 
ft 

a 


'ft 
a 

c3 
CO 


-^ 1 








5.3 

10.6 
9.3 
8.0 


CO coococo 

r-t COi-H 

(N CO 1— 1 ■<*i to 


CO 

1— 1 


00 
00 








00000 


COO CO 




00 00 CO -^ 
—< (M <M 


00 coo "^ c^ 

(M C^ CO Tf 


^ 1 

CO 

1 









6.6 

8.0 

18.6 


COOOO 

CD -^ (N 
fH CO (N CO 




^ 1 

CO 

^ 1 






CO 

CO 


CO CO CO 00 
CO »o CO Tt< 00 


30.6 
44.0 
34.6 











CO 

CO 
1—1 


CO CO CO coo 
00 05 to -^ 

rH (M CO ^ 


CO 
006 

Tt< to 




I— 1 






CO 

to 


coco coo 

CO to CO CO (M 
1-t ,-( c^ CO 


CO 

CO 







CO 






CO 

I— I 


CO CO 
CO 05 (M to 

T-H T— ( CO -^ 






1—1 






CO 

l> 

T— 1 


CO coco 

t^ 00 to Tt^ 

.-H r-l (M Tf 






CO 
CO 






CO 

r-H 


000 

00 CO (M 
(M--^ CO 








1— ( 






4.0 

16.0 
28.0 


00 

TtH to 










(M 






OCOO 

rJH CO CO 
CO 


CO 

to 






CO 


coco CO (M 

10 -^ rH 
,-1 TjH 


CO 

10 







OCO 00 

TfH 10 000 








CO 




4.0 

4.0 
12.0 


ooco 
oi 0610 

T-H (M C^ 








OS 




12.0 

13.3 
32.0 


CO 
oico 

(M >0 








10 




8.0 

6.6 
22.6 


CO 
10 








CO 



CO 


OCO COCO 
000 05 CO 0> 


00 

r-l 

CO 





coco OCO 

CO --H 00 CO 
(M CO 










uO 

to 


COO 
COrt^ 


coco 
oi>cd 

r-l CO 1-H 










05 
CO 


9.3 

13.3 
9.3 


CO CO 
Tj^ 00 










Oi 


coo 00 
(N rfi Tl^ Tji 


CO. 
iO 











00 

10 


oocoo 

Tj^ OOrH 00 



CO 


COOOO 

CO -^ 00(N 
(M CC* J> 


1—1 


CO OCO 

CO Tfi CO 

1-1 <M T-l 


00 



CO CO 
(M CO 


»— 1 

Oi 


1 CO 
1 •> 

1 »-H 


pajisd 

Bd^dU 


□aoo 


h-1-H OOt-H 

1 Oi l^ 
CO "* 10 CO 


05 05 »0 00 
00 1^ CO CD .-1 
10 "5 CO 


CO "^ !>• -^ 

oi> 05 CI CO 
CO "O CO to 


(N COCO i-H 
C^ 05 (N (N 
00 (Ni-H CO <M 


O"-! ->* 10 

t^ (M CO CO 
CO to (N Tt< to 


00 
00 



44 Measurement of Quality in English Composition 



X 



O tf fl 

« w o 

or; S 
wQcc 

2 w ^ 

00 o 22 
^ § g 

s tf 5 

^ w c 

^ go 

^ Q H 

H H f^ 
W S w 

g M H 

fe &: cQ 
o 2 H 

Eh -Q 

«> w tf 
g >; w 

tf 02 W 

<5 H > 

|i) ■ 
W § ^ 

S ^ S 

^ Q ?^ 

I- I 

F « >H 
H W 3 
W H S 

fe ^ ^ 



S 



a> 

a 

O 

02 

*E 
c« 
ft 

a 

o 
o 

IS 

*^ 
CO 

a 

m 


1-H 








o tocooo 

Tfl 00050 


OOCOt^-Tt^ (N 

1-1 t^TtHCO 1 

»— 1 1—1 


CO 


00 
00 






00 

o 


00 00 l> to to 
OO Tt<(M O 

C^ (N f-H tH tH 


CO CO CO (Nt^ 
00O5»>(Nt*H 




CO 






o 

CO 


"^ 00 (N 
(N O CO 

(N (N rH 


to CO to Oi 
00 too CO 
r-\ tH 




CO 








to O to to CO 
COTfi coo 00 

t— 1 (N rH 1— 1 


CO<N 05 
t^(N to 




o 






to 

CO 


(N CO 05 Oi 00 
CO Oi 05 to CO 
y-\ 1—1 


00 CO 
COCO 










(N 


to O t> CO o:> 
CO ■* TtH 05 CO 

tH (N i-i 


05 

to 




O 
O 
CO 






CO 

to 

1—1 


!> CO l> 05 05 

rft C5 "^ CO rH 
1— 1 »-H 1— 1 






1—1 






o 

rH 


0(M05<M 
^ CO 05 (N 

t— 1 T-H 






CO 

CO 






t—l 


COl>- 05 
00 Tti CO 

rH 






T— ( 






t^ CO 

Tf 00 

»— 1 


(M 1 






o 
o 






o '^ CO 
CO (N to 








CO 


O -^ COTt* 
TtH (N to CO 


CO 




o 


O O 00 to 
CO -^ O (M 
(N (N (N tH 








CO 




CO cot^ 


rt< CO 05 
^-00 05 












-* to 05 

t^ coco 


1-1 oq 

00 (M 












00 rt< (M 
O (MrH 


Oi 

05 








CO 

o 

CO 


00 to CO CO CO 
<M 1—1 1—1 


00 

r-i 
CO 


O 
CO 


TtH 00 00 rfH 
(N 1-1 O CO 
(M T-( (M 










to 
to 


<M CO 


to OOb- 

O0Tt< TtH 
t— 1 rH 










05 
CO 


CO to CO 

05 CO 05 
»-H 1— ) I— 1 


Oi 05 
CO CO 

1 












OiOOO 
00 CO CO CO 
(N (M (N (N 


05 










o 

00 

to 


OOOOOOO 
CO O T-i O 
Cq (N 1-1 (N 


o 
CO 


CO CO i> CO 
Oi to 00 


t— 1 

o 

»o 


to to to 

CO oco 

rH t—l I— 1 


00 

to 


1 


1-H 

CI 


o 

I— 1 


pdJBdi 
9ldm 




i> T-H 00 1— I o 
oo> to o t^ 
fO^ toco 


O Oi 05 to 00 
00 t>- CO 05 1—1 
to to CO 


coo'^r^'"* 

O l^05 C^ CO 
CO to CO to 


(N OCOCO rH 

(N 005 (N (N 
CO (N 1-1 CO (N 


0^0"*to 
Ot^(N coco 
CO to (N'"^ to 


00 
00 



Results Obtained from Second Set of Samples 45 



n 

Q 

Q 
P 
•-a 



Samples 


«3 

CO 








I-H Tt< 


<M CO 05 (N 

rH 


t>. CO 


1—1 








(N (N 


rH (N CO TjHt^ 


00 05 


00 

00 








I-H <M 1-H (M 


(M CO CO X rH 


000 

I-H 


CO 








rH (M 


CO Tfl CO rH r^H 
rH 


«o-^ 


•0 








CO rH ,-H CO CO 


Tfl rH CO t^ to 


T^ CO 











I-H rH 1-H C^ 


t^ COCO CO to 


(N b* 




CO 








10 CO CO 10 


00 t^ -* (M 


I-H 


1— 1 
(M 








1-H (M (M 00 to 


I> Tt< rH C^ CO 


to y-t 


CO 
(M 
CO 






(M rH 


^ 00 to CO CO 


CO toCM (N 








(M 






r-i (M 0: 


to 00 t^ Tti (M 


r-i ^ (N 




CO 

1— ( 






rH CO CO 


05 rji 1^ TtH 10 


<N (N r-\ 




CO 






1-1 (N CO 


CO to 00 (M CO 


1-H tJI rH 1-H I-H 




CO 






rH (M CO ^ 
»— 1 


to "ti CO CO 






1^ 

CO 


^ 




^ iM •^ rt^ CO 

I-H 


I>- rH 1-H 






to 


(M Oi 10 

r— 1 T-H 


05 




(N CO 


00:1 00 1-H CO 

I-H 


I-H r-i 






00 

T— 1 

CO 




CO COC^ 05 
1—1 


l>CO I-H 








CO 



CO 




rH (M T-i tH 

T-H 


00 t>- >o 1* C<l 








10 




CO "* CO 

rH 1—1 


10 








05 


1—1 


CO C0 1—1 10 

T— 1 1-H 


1-H CO rH 








01 

CO 


CO(MCO 


"^ CO -^ (N 

T— 1 1— 1 










§ 1 

LO 1 


I— 1 I— 1 Tfi 


00 -^ (N r-l 

I— I 










.— 1 


10 


CO >Ol>> rH 


(N (M .-1 

rH 










J— 1 


rHt^CO (N CO 
1-i r-t 


i—l 1-H 










00 


CO (N 05 00 ":> 
I— I 


^ 










CO 


05 05 »0 10 
i-H 


CO 












CO 


CO Oi CO (N 


rH 










snopi 


30J 


i-H (N CO Tt^ 10 


CO t^oo 05 

I-H 


rH (M CO rf< 10 

I-H I-H fH I-H 1— 1 


COl^ 00 05 

rH ,-H I-H rH C<1 


rH (M CO TtH to 

(N (N (N (N <N 


CO t^ 



-2 

CQ 
(h 

o 

<4H 

o 
H 






46 Measurement of Quality in English Composition 



m 

s 

I 

Z 

H 
o 

^> 

EH ^ 

:< en 

S 2 

X Q OQ 

H '^ ^ 
PQ !2J 






o 
o 

03 






pq 



o 

(6 

m 

PQ 

:^ 

» 



a 
.2 

a 


i 

J 

'E. 
S 

o3 
02 


f-H 








(N (N (NiOOO 


to CO rH 00 »H 
I-H I-H I-H (M 


OS 

rH 


00 
00 








CO ■^ CO CO OS 


oo»o CO CO t^ 

I-H f-H f-H rH 




10 
10 








CO (M I-H Tf CO 

I-H 


CO 00 I> f-H 
I-H I-H (N 




CO 








CO (M(N "^CO 


Tt^ t> OS 
I-H I-H 













CO CO (N 000 
tH 


00 
I-H (M 




1—1 








OS toco '^ CO 

f-H 


OS 







CO 






y-{ 


OtO00(N CO 
I-H fH (N 






(M 






1—1 


to rH 
I-H rH I-H 






CO 
CO 






CO 


oco to 

<N rH I-H 






CO 

rH 






(M CO 1-1 

I-H 


rt< OS 
(M I-H 






(N 






to to 

1-H 








CO 


f-H CO Tt^ 
rH 


CO 

10 


(M <-H (N T-l 
I— 1 


CO 




I-H (N 


CO 00 

tH i-H 








OS 




t-H CO to t^ 


l>00 

f-H C^ 













CO 




I-H 








CO 


u:>(N CO TjH 
1— 1 


00 

rH 

CO 


CO t^ (M 

r— 1 rH 


10 

OS 


i-H 


to o»o 










OS 
CO 


C^CO(M 10 (N 


00 00 

rH CO 










OS 


to 


10 


C^ (N t^ to Tt* 
1— i 



CO 


(M Oi -rt^ rH 
1-1 CI c^co 



10 


CO ot^ 

I-H 


00 

to 


000s 

rH 


t— ( 

OS 


00 


paj-Bd 
afdni 


caoo 


t^ rH 00 — 1 

OS to t-- 
CO '^ to CO 


OS OS to 00 

00i> CO OS rH 

to to CO 


CO '^ t^ ^ 
l:^ OS (N CO 
CO to CO to 


CN oco CO I-H 
(M OOS (M (N 
CO (N rH CO (N 


^ rt< to 

r^ (N CO CO 

CO kO (N -^ to 


00 
00 



-a 

o 



03 






O 

r^ 

a 
o 

A 
Eh 



^< 

Q 

o 
-♦J 

-♦J 

73 

a 



o 

03 
0) 

r£3 



Results Obtained from Second Set of Samples 47 



03 

a 

a 



.22 
'C 

c3 
P. 

S 




■^ 

OQ 
IV 

a 


i-H 








05 05 0i<N 10 


(N t^OS OS C<l 


CO 
CO 


Tt^ Tf* TjH C^ 05 

1-H ,-H 


(M I-H CO CO I-H 

-H CO c^ -^ 10 


00 

00 








CO t>- CO CO 05 


ic CO 00 "* 




t^05 l>t^ i-H 


OS CO OS OS I-H 

I-H CO CO CO Tj< 


10 
CO 








CO Oi -^ t^ b- 

t^ -^ (N 05 ^ 

CO 


coos Tt<(N 

Tf< CO I-H 1-H 
I-H Tf T}< »0 




CO 








CO 05 05 t^ 

b- TtH '^ 05 I-H 

CO 


b- "* CO 

OS I-H CO 













CO CO Ci 10 TJH 

t>. t> rfi 05 '^ 

I-H (N 


Tt< 00 

"^ 00 




10 








OKN CO 00 

I-H 05 t^ 05 OS 

(M .-H CO 


OS 

1-H 






CO 








TjH (M lOCO rH 

"^ (M OS 05 CO 
(M I-H I-H C^ to 














rtH (N OS Tt^ 
(N rH (M oq 






CO 

CO 






CO 


00I>CO 

00 I-H CO 

Tt^ COCO 






CO 

T-H 






05 CO 05 

TJH Tfl CO 
I-H (M 


»o CO 
00 CO 

10 rt< 















(NCO 
(NCO 

T-H CO 


10 

06 
10 






CO 


TflCO rH 

CO 


CO 




CO 






CO Tfl ^ 
■^ Tt^ Tt^ 












Tt< CO (N .-H 


•^ (M 








I-H I-H 


f-iOO 
rt< CO 



10 




CO 








1 


CO 



CO 


(N 05 CO .-1 

1-H CO 


00 

1— I 
CO 


t> .-H CO 

^ t^ 05 
CO T-H (N 


10 




12.2 
48.8 
12.2 










05 
CO 


05 CO Oi CO 05 

rt( t^ Tjl CO -^ 

CO 


05 CO 
COtJH 










1> 


T-H 




00 

10 


Oi 05 T-H CO t^ 

T*l -^ !>. CO OS 

r-H CO 


CO 


CO t^ 10 CO 
05 006 »0 


r-l 


10 


CO ^ »-H 


00 

10 


10 CO 
oi CO 

r-l Tj^ 


I-H 


10 

Oi 
f— 1 


pajBdi 
aidxn 


caoo 


b- 1-H 00 I-H 

OOSiOOt^ 
CO ""I* "5CO 


005CT>iOOO 

00 t>C0C::5 rH 
iO 10 CO 


CO ^ t> "^ 
Ol>-Oi (N CO 
CO^O CO"5 


<N CO CO I-H 
(N OOSCq (M 
CO (N rH CO (N 


I-H -^ 10 

t^ (N CO CO 
C0»O <N -^ 10 


00 
00 



48 Measurement of Quality in English Composition 



o 

QQ 



o 

Q 

CO 
PL. 

> 



w § 

I ^ 



SB 



B P 



CQ 
<1 

!?; 

M 

Pk 
< 

OD 

p 
o 

H-t 

w 
;?; 

» 

PQ 

O 

12: 



02 li; 

l« 

H O 

^§ 
s S 

& «2 
H o 

Q 
I— I ^ 

o 



HO 
> > 



H 

^ 



a 

a 
o 

a 

o 

-% 

IB 

'$ 

o 

a 

m 


I— 1 








t^ h-t>. CO 00 

n^ Tt^ Tt^ ir^(M 

(N (N (N iH ,-( 


CO rH rH CO 
1-H 


1-H 




00 

00 








COCO CO CO to 

»H 05 rH rH rH 
(N rH (N (N i-H 


OO r-< r-^ i-{ C<\ 
(N to Tt^ Tt^ CO 














CO CO lO CO -—I 

iH Tfl Oi 05 t>. 

(M (M (M i-i 


COCO (M 
tO(NC0 






CO 








CO CO CO CO rH 
r-l rJH Tin 05 !>• 
(M (N (N tH 


CO CQ TT* 
05 CO rH 
rH 






O 








CO CO CO 00 CO 
-H rH T^ C^ O 
(M (N (N rH rH 


CO to 

o 






I— 1 








to CO CO CO rH 
rH I>. rH Oi "^ 
iH tH (N rH 


lO 
rH 






o 
o 

CO 






to 

Oi 


CO CO OOrH . 

OIXM 00 • 

rH rH ,— 1 








T-H 

(N 
CM 






to 

Oi 


CO CO tH CO 
O t^ Oi o 

rH tH rH 








CO 
CO 






CO 

rH 


to iH rH 

I>iO 








CO 

o 

T— 1 






CO CO rH 

"* to 05 


rH 








O 

o 


CO T— 1 
l> to 

t— 1 




CO 


lO CO r-( 
C5 1— 1 CO 




CO 


CO lO CO rH 

•^ Oi •^ Oi 
<N (N (M 




CO 




to CO 


CO CO CO 
tOOO 

rH tH rH 










05 




uO CO CO tH 


CO • 














CO 


CO 

o 

rH 










CO 

o 

CO 


CO CO CO rH 
I> TtH 1— 1 CO 

i-H (N (N 




00 

T-t 

CO 


T— 1 1-H 1— 1 

l>. rf* 00 
1— I 






CO 


CO lO CO 

iH I— 1 












05 
CO 


CO CO CO rH CO 
Ttl T-l rJH lO Tt^ 
(N (N (M (N 


CO 












05 


CO 

1— ( 




>o 


CO CO rH rH CO 
Th •^ Tt< to 05 
(N (N rH rH 




CO 


00 




T-H 


Tt^ CO 1-4 
(N rH rH 




00 


00 '^ 
r-( 




T-H 


00 

1— I 




pajBd 
ajdu] 


moo 


l> rH 00 »— 1 O 

co-^ io CO 


O 05 05 to 00 
OOt-CO 05 1-I 
»0 to CD 


CO O ^ t^ ''f 

O t^05 (N CO 

co»o COtO 


(N O cOCOrH 
(M O 05 (N <N 
CO IM rH CO C<J 


O --I O "vf^ to 

o t^ c^ CO CO 

CO to (N ^ "5 


00 
00 





Results Obtained from Second Set of Samples 49 

A preliminary examination of the results obtained with the 
set of twenty-seven samples indicated that two other samples 
should probably have been added. These samples, 519 and 520, 
were added and twelve individuals judged this set of twenty- 
nine samples. The distribution of positions assigned the vari- 
ous samples and the number of " better '' judgments are given 
in Tables XIV and XV. 

After two hundred and two individuals had judged one or 
the other of the sets of samples, it was decided to form the 
scale. Twenty-one of the samples were common to each of the 
sets. The number of " better " judgments for these samples 
was obtained by combining the data bearing on these samples 
from Tables IV, VII, XI, and XV. The " better " judgments 
were reduced to per cents and the differences between the 
various samples determined and given in Tables XVI and XVIII 
respectively. The samples composing the scale as given in 
Section 2 were selected as a convenient series from the set 
given in Table XIX. 



r 

^^^:^ 



^^ 



^^^' 






50 Measurement of Quality in English Composition 



Q 

PQ 
to 



CQ 

2 



H 

iz; 
> o 

X g 
PQ O 



g 



H 

O 

02 
OQ 

< 

m 
O 



OQ 

o 

H 
O 

o 

M 

H 

n 
i-i 

H 

CQ 





CO 


rHfH C^CO CO »H rH 




i-H 


(M ^ i-H C^CD 




00 
00 


^ <NrH CO (NCO 




CO 


1-H fH (N (MCOCO 




1—1 

10 


i-l (N 1-H (N (M CO i-H 







'— t »H (M C^ 1— t Tt< I-H 







f-H C<|i-HC^CO rHi-Hi-H 







i-H 10 f-i <N f-i (N 




1— 1 


t-( Tt< »-H i-( CO c<» 




CO 

(M 

CO 


C>l 1-H r}1 I-H 1-H C^ r-t 







CO 


(N(N "* i-H (N 1-1 






1-H I-H CO (M 01 »-H I-H I-H 




(M 

CO 


f-i (N (N (Ni-H CO 


1 


CO 


(N""^ (N (N i-H^ 


Oi 

T-H 

10 


i-H CO CO 1-H CO 1-1 


CQ 


CO 


^ (N Ol »-H f-H f-H 1— 1 





10 


I-H d >-l I-H CO i-l rH C<l 




Oi 


CO 1-H Tf 1-H I-H C<1 




00 
1—1 

CO 


»-l 1-H r-l CO CQ »-H I-H Ol 




CO 


CO 


rH (N (N i-HCO CO 




10 
01 

uo 


T-H (M 01 >0 I-H I-H 




05 


(N f-H kO(M 1-H I-H 




Oi 
CO 


1-H toco (N ^ 






00 

10 


1-H I-H 01 CO rH rH 




1— 1 


10 


I-H T^ Tj< 01 rH 




1—1 
Oi 


Tt< CO CO Ol 




00 
10 


I-H Ol -^CO 01 





CO 


CO -^ CO 1-H I-H 




l> 1 

1 00(N <N 
CO 1 


euoT^^i 


SOJ 


»H01C0"^"5 «OI>00050 r-H Ol CO ""l^ U5 COr^OOOO 1-H 01 CO TJK 10 COb-OOOi 
r-n,N>.j-=r- ^ ^ ^^^^^ ^^^^C^ OIOIOIOIOI OIOIOIOI 



Results Obtained from Second Set of Samples 51 



H 

o 
o 

CO 

03 
00 

-*-3 



a 

o 
ft 

o 



•^ fe 



CO 


w 

ft 

a 


u 

Id 

■^ 
"ft 

s 

CO 


1— 1 










>-H 1-H CO I-H 


Tf CO lO 


00 

GO 








I-H 


i-l Tfi rti CO 


CO CO 


10 

CO 








I-H 


(N Tf rt* t^ 10 


Ci 


CO 








I-H 


(N.-H (Nvo Tf 













f-H (N 


^ 10 j>r^ 




10 








»-H 1-H I-H 


C^ "^CO 













(M (Ni-KN 


CQ to 







CO 








I-H I-H I-H 1-H CO 


CO 






(N (N i-H CO CO 


CO 


CO 10 CO CO 


CO 

OS 


(MiO CO 





(M 






I-H 1-H rH 


COCO 






CO 






CO coco 


»Ci 






CO 


CO -^ CO 


05 

»— 1 
10 


->* ^ 10 


CO 




CO 


(NCOCO 








05 




iC 


00 I-H 

T-H 













CO 


CO 








CO 

CO 


Tf<CO 


00 

I— 1 

CO 


,-110 


10 


Ttl 


CO 





C0 05 










05 





CO 












00 


i-H ^<NO 


CO 


CO 0000 

T-H 1— ( 


1— 1 




(N<N r-l 


00 


CO CO 


1—1 
Oi 


(N 


pajBdi 
8|dun 


noD 

3S 


l> T-H 00 .-I 
005^01^ 

CO Tf »0 CO 


05 05 1000 
00 t>- CO OS i-i 

io »c CO 


CO Tfi b- 

Ot> 05 (N .-H 

CO iC CO 10 


"* (N OCOCO 
CO C^ 05 <M 
»OC0 (M t-H CO 


»-H I— 1 

(M (M t^ 'M 

(M CO iO »0 CM 


'^ 'O 00 
CC CD CO 
Tf >r5 Tt< 



52 Measurement of Quality in English Composition 



o 

H 

n 



0) 

a 

O 

CQ 
*C 

03 
ft 

a 

o 
o 

*^ 
J 

'ft 

i 

CQ 


I— 1 








CO 1— 1 05 O 
CO U5CO00 


05 05 
00 00 


oo 

00 






CO rJH 


00 OiO^C 
Tt< kO COOO 


00 


CO 






coo 

1-H Tj< 


CO Tt<(NO 
CO !> COOi 




CO 






C0U3 


Oi 00 I> "5 
C0iOI>l> 




o 






l>»O00 
CO "5 CO 


l> OS 

CO o 

»— 1 




t— ( 






CO .-1 CO 
(NTf CO 


o 

CO 




* 

o 






OO 

CO 00 


CO 

05 




o 
o 

CO 






■^ 1— ( O T-H CO 
CO <M Tt< t^05 










00 


OIXNO 
rt^ CO^ 00 






CO 
CO 




CO 


rH OOCO 

COCO CO 






CO 
I— 1 




CO »OiO 
(M (N kO 


00 Oi 






o 
o 




CO T-H 00 

rH COt^ 


CO 

00 






CO 


0(N rfi 

r-H(N 00 


CO 


(N COOS CO 
i-H T-t CO -^ 


T— 1 


00 (N 
"^CO 


CO 


COI> 


COCO 












o 








o 


00 rH ,-1 


CO 

o 

CO 


O5 00CO 
CO 00 


00 

1—1 

CO 


rHOO»-H 
CO 








CO 


paj'Bdi 
aidra 


noo 


t>-o lo ooco 

OOO 05 .-H o 
O "O »0 CO CO 


* 

O ^ t^ 05 ^ 

t^Oi (N t-i CO 

"5 co»o»o 


(M O coco rH 
<N005(M(N 
CO (N tHCO (M 


* 

OOi-HO Tti 
O<Nl>(NC0 
COiOiOlN -^ 


iCOO 

CO 00 



ft 

0) 

o 



a 

o 

Ci 
CQ 
QQ 
(U 

bO 

;=! 
o 



o 

CO 

o 



-^ S 
O — 

CQ -tJ 

(K a 

4J ft 

(3 ^ 

.« CQ 



M Q) 

t-* ft 

^ a 

OS 55 



lO 



O) CQ 
O O) 

"fl bO 

g^ 
c ••-» 

CQ «*H 

O) O 

OD S 
jt ^ 
* fi 

® 

-a 



Results Obtained from Second Set of Samples 



53 



> 
X 



to 

O 
Q 
P 

•-5 
O 

a 

PQ 

O 
H 

K 
H 

E^ 
O 

;?: 
O 

Ph 
O 

Q 

O 

P 
Q 
» 

WP5 

PQ 

H 



J? 

M 

o 

00 

O 
Q 
P 
•-5 

O 

n 
1^ 
P 
^2; 

w 
m 
H 



PQ 



S 
O 

CO 

a 

o 
o 

o 

a 

02 


1—1 






00 ^ 

r-id 

I— 1 »— 1 


17.8 

25.3 
19.3 
39.6 


1— t 1-H 


00 
00 


1 

1 




CO CO 

i>d 

1-1 (N 


23.8 

29.2 
29.7 
42.1 


to 

00 


CO 






-^ 00 

CO d 

1—1 


16.3 

36.6 
30.7 
44.6 




CO 






00 CO 

CO to 


CO !>. 1-1 1-1 
Oi00 00l> 

1-1 ca CO CO 




o 






CO (N t^ 

ooi>co 

r-H (M CO 


33.2 
54.0 




T— i 






i-H d oi 

1-1 C^ CO 


d 




o 

CI 






t^CO 

C55 Cft 
(M CO 


to 




o 
o 

CO 




d 


00 -^ 00 (M to 

CD O Oi to !>. 

1—1 1—1 1—1 CO Tti 










05 
CO 


00 CO i> CO 

Oi OO to Oi 

r-H 1-1 (M CO 






CO 
CO 




CO 

l> 

I— 1 


C^OO CO 

dood 

CO —1 CO 






CO 
f— 1 




rfi Tt^ CO 

1-1 <N t>^ 

1— 1 1— 1 C^ 


1—1 1— 1 
(N to 






o 
o 




CD rtl CO 

i>ico6 

1— 1 CO 


CO 






CO 


5.0 
10.9 

41.6 


CO 

to 


05 OJ CO 00 




^ d 

(M CO 


CO 


^ CO 

1—1 1-^ 


i> CO 

(M CO 








05 


'^ COI> 

00 CO d 

(M CI 


lO 

d 

lO 








O 
U5 


4.0 
20.3 
25.3 


CO 
O 

CO 


lO 00 CO 
rJHo6<M 

1-1 Tt< 


00 

1-H 

CO 


^ O (M 
' Tji d 

CO 


to 

Oi 
lO 


O 05 

r-^d 

r- 1 


o 

00 


1— ( 


pajBdx 
axdtui 


noo 


t>0»0 00 CO 
O 00 Oii-t o 
CO ^ 'O CO CO 


b-a> (M rH CO 

to CO to to 


(MOCOCO 1-1 
(N 005 Ca (M 

CO (N r-H CO C^ 


■X- 

O O -H O "^ 
O (N t^ (N CO 


to 00 
CO 00 

to -^ 



> 
X 
o 

I 



o 

o 

a 

m 



54 Measurement of Quality in English Composition 



n 



> 
X 



> 
O 

o 

o 

Eh 
Z 

O 

o 

S3?, 

§« 

7 <) 






CO 

§^ 

CO Q 

g CO 
P (O 
O H 

« £ 

W Q 

tx 






ID 
c3 

a 

.2 

fl 
o 

.S2 
*C 

a 

o 

IB 

CD 
0) 

a 

m 


1— ( 






CD CD 
t^OO 

r-< I— ( 


l> <3i 05 

CO Oi(M 

T-t tH 


00 
00 






-^ 

I— 1 1— 1 


CD r-H Oi 

oot>. 

rH 


CO 






CD CD 

(M i-H 


to rH to 
"* to b* 

rH 


CO 






(MOi 

1-H 


Oi CO to Oi 
(NOOtJH Tti 


O 






CO Oi CD 


CD 


1— 1 






05 Tt^ CD 

t^(N CD 

y—^ 1—1 


Oi 


o 


05 05 
l>CO 


o 
o 

CO 




CD 
00 
rH 


(MCD CDCD 
Tt^ 00 (N to 1 

»— 1 1— 1 i-H 




1— 1 




T— I 

CD 
rH 


CD rt^I> Oi 
C^ CO Oi CO 

I-H I-H 




CO 
CO 




O 


r^ i-H Oi 

l>CO CO 

T— ( 




CD 




05 rH O 
l> t> 05 

y—< rH 




CO 




O 

o 




0:1 rH CO 

to -^ 

(N i-H 


00 1 




CO 


-^ CO (N 

Tt^ 00 CO 

(N rH 


CO 
to 


C^ a> 05 rH 

CO 0<M T-H 

(N (N rH rH 


* 

Oi 

T-H 


to to 

T— 1 


CD 


rH rH 


00 t^ 
00 "^ 






05 


"^ Oi Oi 1 


o 


1 O Tt^OS 

1 CD (N Oi 


CO 

o 

CD 


1 (N r-H 00 

1 toco (M 


00 

1— 1 
CD 


1 Ob- 

1 CD t^ 


to 
to 


1 1^ 
1 00 

1-H 


o 

00 

to 


1 


pajBdt 
a[cira 


MOO 
«3S 


t^ O to 00 CO 

OOO C55 rH O 

(X) to to CD CD 


* 

"* l> Cni Tfl 

Ir^Os (M .-H CO 
to CD to to 


(M OCD CO -H 
(M Oi(N (N 
CO (N i-H CO (N 


* 

I-H -^ to 00 
0(MI>(NCO CDOO 
CO tOtO (N Tt^ to Tt< 



X 



Eh 



02 



Results Obtained from Second Set of Samples 



55 



TABLE XIX 

The Samples Selected for the Scale, the Differences Between 

Adjacent Samples and the Difference Between Each Sample 

AND Sample 580 for Each of Two Groups of Judges 





Group of 202 judges 


Group of 41 judges 


Sample 


Differences each 
from preceding 


Differences 

from sample 

580 


Differences each 
from preceding 


Differences 

from sample 

580 


580 
595 
618 
94 
519 
534 
196 
221 
571 
177 


183 

77 

109 

105 

111 

90 

97 

66 

99 


000 
183 
260 
369 

474 
585 
675 
772 
838 
937 


173 

81 

141 

* 

246t 
91 
91 
41 
71 


000 
173 
254 
395 

641 
732 
823 
864 
935 



* This sample is not in the set judged by this group, 
t This is the difference between samples 94 and 534. 



Section 9. Sufficiency of Judges 

No claim is made that the values given in the scale are ab- 
solutely accurate. Variation among the judges was very great, 
and to make a perfect scale would require the services of many 
more judges than it was possible to secure for this study. The 
scale is accurate enough to be of very great practical value in 
measuring the merit of English compositions written in the 
upper grades of the elementary school and in the high school. 
The scale will also serve as the basis of future efforts in this 
direction, and it can be refined and perfected part by part. 

The scale was based on the judgment of two hundred and 
two individuals. Several checks were employed to test the 
accuracy of the work done by these judges. Table XIX shows 
the differences between adjacent samples and the absolute values 
for the samples of the scale. One part of the table shows the 
results for all the judges and the other part shows the results 
obtained from the group of judges described on page 40. The 
differences in the absolute values in the two results are surpris- 
ingly small. In no case do the values vary by a complete step 
in the scale, and the value of the best sample is almost the same 
in both results. This similarity of values indicates that the 
values in the scale are reasonably near the values that experts 
in English composition would assign to the samples. 

If the scale were accurate, the values of the samples would 
remain the same no matter how they were obtained from Table 
XVIII. Thus the difference in value between samples 580 and 
6x8 in the scale is (183 + 77) 260, and in this case the table 
shows the same difference when sample 580 is compared directly 
with sample 618. The agreements that exist between the results 
obtained by these methods are seen in the following tabulation: 

Difference between samples 580 and 618 in scale is 260, in Table XVIII 260 
« a « 595 « 94 « 186 « 204 



tt 


u 


u 


94 " 534 


u 


226 


a 


209 


a 


u 


11 


534 " 221 


(I 


187 


u 


161 


u 


u 


u 


196 " 571 


il 


163 


u 


179 


it 


u 


u 


221 " 177 


u 


165 


u 


186 



S<5 



Section io. The Quality of Judges 

The question of what individual or group of individuals rep- 
resents the best judges of English composition is a difficult one 
to answer. The data at hand do not show any marked dif- 
ferences between the various sets of judges so far as uniformity 
of judgment is concerned or faithfulness to the proper order of 
the samples. In order that some definite conclusion might be 
reached concerning the reliability of judges, the following scale 
of penalties for placing samples in the wrong order was de- 
veloped. Where the differences between samples are very great 
the penalty for misplacing has been made correspondingly large. 
Thus the difference between samples 627 and 534 is 129. If 
the judge considers sample 534 better than sample 627, he is 
guilty of a much greater error than he would be in placing 
sample 571 before sample 220, where the difference is only 15. 
In the latter case the penalty was arbitrarily fixed at i while 
in the former it was placed at 6. 



TABLE XX 

The Differences Between Certain Sample Compositions and the 

Scale of Penalties for Placing Samples in Wrong 

Order by Individual Judges 



Samples 


Ascertained 


Penalty for 


compared 


differences 


wrong order 


94 


with 


627 


47 


2 


627 


« 


534 


129 


6 


534 


u 


322 


32 


2 


322 


u 


200 


43 


2 


200 


u 


196 


20 


1 


196 


u 


323 


66 


3 


323 


it 


221 


39 


2 


221 


« 


300 


10 


1 


300 


a 


220 


64 


3 


220 


« 


571 


15 


1 


571 


u 


434 


45 


2 


434 


u 


565 


20 


1 


565 


u 


488 


5 





488 


H 


177 


22 


1 



57 



58 Measurement of Quality in English Composition 

The following tables show the penalty each judge in each 
group received and also the median penalty for each group. 



TABLE XXI 

The Penalty for Wrong Order Received by Each of the 

Individuals in the Several Groups of Judges 



Group of 74 


Group of 75 


Special Gr 

o 
bC 


oup of 41* 


bC 


Oh 


o 

bC 


>> 


o 

bC 

:3 

•-5 


>> 


bC 


Is 
a 


(D 

fin 


1 


5 


39 


8 


1 


7 


39 


8 


1 


7 


2 


5 


40 


6 


2 


6 


40 


7 


2 


13 


3 


6 


41 


8 


3 


11 


41 


9 


3 


10 


4 


19 


42 


11 


4 


5 


42 


5 


4 


13 


5 


7 


43 


17 


5 


10 


43 


10 


5 


8 


6 


9 


44 


5 


6 


12 


44 


6 


6 


8 


7 


10 


45 


10 


7 


9 


45 


9 


7 


8 


8 


8 


46 


7 


8 


10 


46 


11 


8 


6 


9 


5 


47 


9 


9 


9 


47 


9 


9 


10 


10 


12 


48 


12 


10 


12 


48 


6 


10 


13 


11 


6 


49 


8 


11 


15 


49 


9 


11 


9 


12 


15 


50 


8 


12 


11 


50 


9 


12 


16 


13 


10 


51 


8 


13 


8 


51 


9 


13 


5 


14 


10 


52 


11 


14 


5 


52 


4 


14 


14 


15 


7 


53 


11 


15 


10 


53 


9 


15 


9 


16 


9 


54 


10 


16 


10 


54 


14 


16 


11 


17 


9 


55 


6 


17 


16 


55 


3 


17 


2 


18 


5 


56 


7 


18 


11 


56 


11 


18 


13 


19 


11 


57 


12 


19 


14 


57 


13 


19 


12 


20 


8 


58 


10 


20 


9 


58 


7 


20 


10 


21 


12 


59 


15 


21 


14 


59 


7 


21 


12 


22 


7 


60 


12 


22 


8 


60 


10 


22 


5 


23 


9 


61 


12 


23 


3 


61 


11 


23 


7 


24 


14 


62 


12 


24 


9 


62 


6 


24 


9 


25 


4 


63 


10 


25 


16 


63 


5 


25 


11 


26 


4 


64 


6 


26 


16 


64 


8 


26 


11 


27 


12 


65 


7 


27 


10 


65 


13 


27 


7 


28 


14 


66 


7 


28 


13 


66 


4 


28 


6 


29 


9 


67 


11 


29 


13 


67 


8 


29 


5 


30 


4 


68 


11 


30 


8 


68 


11 


30 


13 


31 


15 


69 


12 


31 


9 


69 


14 


31 


5 


32 


12 


70 


6 


32 


8 


70 


15 


32 


6 


33 


7 


71 


8 


33 


5 


71 


8 


33 


2 


34 


10 


72 


10 


34 


7 


72 


9 


34 


7 


35 


11 


73 


10 


35 


6 


73 


13 


35 


11 


36 


6 


74 


11 


36 


5 


74 


15 


36 


14 


37 


17 


75 




37 


3 


75 


8 


37 


6 


38 


7 






38 


9 






38 





* Data were not calculated for four of the judges in this group. 



The Quality of Judges 



59 



TABLE XXII 

Distribution of Penalties Given in Table XXI 

The table reads as follows: In the group of 41 each of two judges 
received the minimum penalty, 2; in the group of 75 each of three judges 
received a penalty of 3; in the group of 74 each of three judges received 
a penalty of 4, etc. 









Special 


Penalty 


Group of 74 


Group of 75 


group of 41 


2 






2 


3 




3 




4 


3 


2 




5 


4 


6 


4 


6 


7 


5 


4 


7 


9 


5 


4 


8 


9 


7 


3 


9 


6 


12 


3 


10 


10 


7 


3 


11 


8 


9 


4 


12 


10 


3 


2 


13 




5 


5 


14 


2 


4 


2 


15 


3 


3 




16 




3 


1 


17 


2 






18 








19 


1 






Median 


9.8 


9.8 


9.6 



Table XXII shows the distribution of penalties received by 
each group of judges. It will be recalled that the group of 74 
represents the individuals who judged the set composed of 
83 samples. These persons were largely graduate students of 
education. The group of 75 is composed of those who were 
selected for their general ability as competent judges of Eng- 
lish composition. The special group of forty-one is composed 
of those who may be said to be expert judges of English com- 
position. The table of distribution of penalties shows no im- 
portant differences in the three groups. The medians for the first 
two groups are 9.8 in each case while the median for the last is 9.6. 

By referring to Table XXI, the individuals whose penalties 
are at the extremes of the distributions may be found and 
from data in the possession of the author their identity may be 
determined. In the special group of 41, judge number 17 re- 
ceived a penalty of 2. This individual is a psychologist, a clear, 
convincing writer and an editor of large experience. Were his 



6o Measurement of Quality in English Composition 

name mentioned here it would be that of a man who has an 
international reputation. The largest penalty received by any 
judge in this group was i6 by judge number 12. All that was 
said regarding the ability of judge number 17 could be said of 
this judge, except that his field is philosophy instead of psy- 
chology, and he has not had a wide experience in editing. These 
individuals would be considered equally competent as judges of 
English composition, yet their judgments show the widest diver- 
gence found in this group. Judge number 23 in the group of 75 
has a penalty of 3. This individual is a woman who reads a 
great deal of the best literature. She is not a teacher, and ex- 
cept for occasional papers prepared for a study club, she does 
not write. Judge number 17 in this same group has a penalty 
of 16. This individual is the author of several novels and 
descriptive works. He has had considerable experience as a 
newspaper reporter and was for some time the editor of a 
paper. Judge number 6 in the group of 41 is the author of 
several texts largely used in the English work of the high school. 

Section ii. The Use of Mixed Types of Composition 

Two methods of selecting samples from which a scale might 
be derived are possible. The first is to take only compositions 
that would be classified as narration or as description. The 
second method is to pay no attention to such distinctions ; rather 
to make the effort to include all the various types. Either 
method is good. In this study the effort was made to include 
all types except poetry. It was believed that in this way the 
scale could be made of more value to the teacher. In the upper 
elementary grades and in the secondary schools the distinction 
between narrative, description, and argument is not an impor- 
tant one. Thus the scale will measure the actual products ob- 
tained in these grades. The objection may be offered that it 
is impossible for a person to compare a sample of narration 
with a sample of description, that we are trying to compare 
entirely different things. The answer is that people actually did 
do it. Of the four hundred and fifty people who have judged 
these samples not more than three have offered any objection 
on the score that they could not compare the samples. 



I 











rr, -.^''T?::: ^'^>^Ki*^^• 



,'/►.' 2.,- 

wm 











C-Vj."..